Have an idea?

Visit Sawtooth Software Feedback to share your ideas on how we can improve our products.

Segmentation analysis from individual utilities


I used CBC HB and SMRT to genereated the zero-centered diff utilities at an individual level and would like to use the values for segmentation analysis.
If I segment my respondents (e.g. male vs. female) and take the average utilities / importance, is there a way to test for the quality of the segment / statistical significance of the average ?
What I mean is, is there a way to check whether the respondents from my segment truly have similar preferences, and whether the average calculated is actually a good measure of the segment's preferences ?

Thank you !
asked Jun 22, 2014 by anonymous

1 Answer

0 votes
This is a good question and perhaps a little more complicated than you might guess.  The short answer is "yes," with zero-centered diff utilities you can compare segments to see if their utilities are significantly different or not.   The quick and (only slightly) dirty way to do this is to just treat the utilities as you would any other measure and run a statistical significance test on them.

Use a t-test if you compare two groups and an analysis of variance if you test more than two; moreover, as you are likely testing the significance of multiple attribute levels you may want to use a multiple comparisons test that corrects for the "false discovery" that can occur when you apply the 5% error rate for one test (i.e. 95% confidence) to multiple tests.  For this false discovery correction I like the Benjamini-Hochberg procedure, a nice description of which you can find on Wikipedia.  

I mentioned above that this kind of traditional stat test is a good way to test but if you want to be really rigorous you could go into the "draws" files and run a proper Bayesian statistical test.  This is a bit more work, however and I suspect your needs will be very nicely met with the traditional stat test.

Either way, if you run these tests and find significant differences you can conclude that your segments have differing preferences.  Concluding that they have the SAME preferences is not the flip side of this, of course, as the size of differences you are able to detect is dependent on your sample size.
answered Jun 22, 2014 by Keith Chrzan Platinum Sawtooth Software, Inc. (93,025 points)
Thanks a lot for your answer, I will look at the options you suggested.
I do have one concern though. The solutions you mention seem to be appropriate to verify whether two groups really do have distinct preferences.

What I'm really trying to do is specify if all the respondents within the SAME group selected have similar or divergent preferences. I know I could measure the SD of the utilities / importances around the mean, but is there another / more complete way to do that ?

Thanks again for your help.