Have an idea?

Visit Sawtooth Software Feedback to share your ideas on how we can improve our products.

Maxdiff with large number of items

I have a list of 40 items. I order to eradicate respondent fatigue and to reduce the number of screens I am thinking of doing an express maxdiff randomly selecting 20 items (out of 40) and conducting a maxdiff out of 20 items. Teh process is detailed out in the sawtooth paper page 59 under topic "Using Maxdiff for Evaluating Very Large Sets of Items".
 Here we have to create block designs by allowing the option "Individual Designs Lacking Connectivity". My questions
i) Why should we not allow connectivity between individual designs?
ii) After forming individual maxdiff designs which results to showing 12 screens per respondent with 5 items each (each respondent evaluating 20 items out of 40 here), I am exposed to the warning of versions lacking connectivity when I try to combine the designs. Is the warning ignorable?
iii) Such a design have any impact when I try to HB estimate the utilities? should I try any other method for generating utilities?

Please help.
asked May 11, 2018 by anonymous

1 Answer

0 votes
If you're using the Worth and Wolfrath strategies (Express and Sparse MaxDiff) then no individual's design will contain all the items in your study, so the designs cannot exhibit connectivity.  That's why you turn the setting on to allow designs without connectivity and that's why you'll get the warning later.

In answer to your third question, HB works very well:  in this paper (https://www.sawtoothsoftware.com/support/technical-papers/175-support/technical-papers/maxdiff-best-worst-scaling/1493-a-parameter-recovery-experiment-for-two-methods-of-maxdiff-with-many-items) we show the results of running HB on data from Express and Sparse designs.  That paper used artificial data.  At our recent Turbo Choice Modeling event we showed very similar results for a study among human respondents, so the finding appears to be robust.
answered May 11, 2018 by Keith Chrzan Platinum Sawtooth Software, Inc. (115,350 points)