Have an idea?

Visit Sawtooth Software Feedback to share your ideas on how we can improve our products.

How to test statistical significance from MaxDiff (BestWorst-Scaling) factors?

Hello experts!
I am currently writing my Masters Thesis and was luckily enough to get to know Sawtooth software for my preference problem.

I set up a survey where participants had to answer 18 BestWorst questions. The attributes list contains 30 attributes, while I choosed to show every item 3x. The question design is random (every participant received a different question sample)

For now, I manually did counting analysis and rationale scale analysis to rank my items.
I am planning to integrate the Sawtooth "TURF" analysis to verify my results (in case of population reach for the "best-scaled" items)
Now I am wondering on how to test statistical difference between those items, since I have no classical categorization questions (atleast not enough as I have only 90 respondents).
I read alot about the logit-model or HB-testing, but honestly I don't understand these testing mechanisms for my case.
My thoughts were to apply simple t-tests for the rescaled scores delivered by the MaxDiff-analyzer. I could sup-sample my data into those participants with a rescaled-score of 100 and those with sum_rescaled_scores ><100. What do you think about this?
Gladly for all your effort, I remain with best wishes from Germany.
Thanks in advance, Tobias
asked Sep 2, 2019 by Tobias Henke (160 points)

1 Answer

0 votes

Statistical testing makes sense only in the context of some hypothesis you want to test.  It matters whether you are trying to show that utilities for items are significantly different from 0 or that the utilities for some items are different from those of other items or whether you are trying to compare utilities across some subgroups.  It sounds to me like your need is the second of those I listed, showing that some items' utilities are different than other items' utilities.  Is that correct?
answered Sep 2, 2019 by Keith Chrzan Platinum Sawtooth Software, Inc. (95,675 points)
Well..not that easy, or maybe?