Have an idea?

Visit Sawtooth Software Feedback to share your ideas on how we can improve our products.

When to use relevant items Max Diff estimation default option?

Hey everyone,

at first, thank you for integrating the relevant item MaxDiff!
After watching your video "Dynamic MaxDiff" and reading the Lighthouse documentation on this topic, I am wondering about the default option on treatment of missing items.

Question Q1 which leads to the constructed list for MaxDiff is "Which of the following flavors of ice cream would you consider eating?". I understand why you would treat unselected flavors as inferior to selected flavors (method 2 of your documentation) or totally unacceptable (method 3). I do not understand why the missing items of a single respondent should be inferred from the remaining sample (method 1 (default)), as the respondent stated the flavor as unconsidered.

I can think of scenarios like "There is a pool of 50 flavors and respondents get to evalaute a MaxDiff of 20 flavors randomly selected in the background." where I would use method 1. I can not think of a scenario where respondents actively select items from a list to then use method 1. I feel like I am not using the information given by this active select then.

Can you explain to me which method you would recommend for the example shown in your video and why?
Or do I just misunderstood method 1?
asked Jan 21, 2021 by SimonV (215 points)

1 Answer

+1 vote
Best answer
The default setting for estimating utilities for constructed list MaxDiff is "missing at random", and this should only be used if the way the constructed list was created involved randomly selecting the items to include (such as Express MaxDiff).  Also, if you are using Bandit MaxDiff, the proper setting is also "missing at random", though this doesn't seem as obvious (and it certainly relies on the notion that aggregate logit is the recommended utility estimation approach for Bandit MaxDiff).

If you are asking the respondent a screening question that involves preference, such as in our Ice Cream example, you should use either options 2 (missing items are inferior) or 3 (missing items are unavailable) for utility estimation.
answered Jan 21, 2021 by Bryan Orme Platinum Sawtooth Software, Inc. (198,315 points)
selected Jan 26, 2021 by SimonV