Have an idea?

Visit Sawtooth Software Feedback to share your ideas on how we can improve our products.

MaxDiff Analyzer : Significance testing and other questions


I used the Zero-centered raw scores for the online MaxDiff Analyzer. While Im exploring this tool to analyze my MaxDiff Data. I do have a couple of questions that I couldn't get clarity on while going through the documentation.

1. What is the difference between the rescaled and Probability of choice? Which one should be used ? If the probability of choice suggests that the probability for item 1 is 89%. IS it safe to say that the respondent would choose Item 1 as "Best" when referenced against otehr items 89% of the times?

2. I do have to segment the data by 6 categories. How can I test significant differences amongst scores? Again, should i be using probabilities here or scores?

3. Will TURF be a good option to see how groups of these items perform? or can anyone recommend a different approach?

New to Sawtooth and still trying to get my basics right! Any help would be appreciated. Thanks in advance!
asked Apr 5, 2021 by Karuna Tandon (300 points)

1 Answer

0 votes
For question 1, the documentation goes into a little more detail about the rescaling methods here: https://www.sawtoothsoftware.com/help/analysis/manual/index.html?standard-rescaling-methods.html.  Generally we see the rescaled scores that sum to 100 used to report to others, as the probability of an item being chosen as best in kind of an average task is a little harder to explain.

For question 2, we have a chapter available from the book Becoming an Expert in Conjoint Analysis that goes into pretty good detail on stat testing choice models here: https://sawtoothsoftware.com/resources/technical-papers/statistical-testing.  But, the app itself has a short section on this as well in its help: https://www.sawtoothsoftware.com/help/analysis/manual/index.html?testing-for-differences-betwee.html

For #3, TURF is an interesting alternative to look at bundle optimization instead of just taking the top items on average, though there are a few ways to classify if someone is "reached" or not.  They are discussed here: https://www.sawtoothsoftware.com/help/analysis/manual/index.html?turf.html.  My personal opinion is that the first choice is probably a bit too extreme (though still valuable), and while the weighted by probability approach isn't really strict TURF, I like it a little more.  The anchor approach is the most legitimate, but requires that you asked specific anchoring questions in the survey to use.
answered Apr 5, 2021 by Brian McEwan Platinum Sawtooth Software, Inc. (56,045 points)