MaxDiff: can you set up 3-way prohibitions?

Question

I’ve got a MaxDiff exercise where the items being evaluated are 12 different insurance plans.  Each plan is somewhat complex (premium, deductible, amount of coverage, etc.), but they are 12 fixed plans with fixed features (so this isn’t a conjoint problem).  Because they are complex, I would like respondents to only have to evaluate three at a time.

The hitch is that I’ve got 7 “premium” plans (higher premium and higher coverage) and 5 “minimal” plans (low premium with minimal coverage).  At the end of the day, our client would like to pick one or two of the “premium” plans and one or two of the “minimal” plans to go to market with.

In real life, people will be choosing from at least one “premium” and one “minimal” plan so the thinking is that the MaxDiff choice sets should adhere to that constraint as well to avoid cross-effects (i.e. choice of "premium" plans could be influenced by the presence or absence of a "minimal" plan, or vice versa).  Is there a way to accomplish this?  It doesn’t seem like I’d be able to accomplish this with simple prohibitions because any particular pair of plans would be allowed to be seen together.  It’s certain 3-plan sets (all three "premium" or all three "minimal") that would not.

Answer 1

Hi segrin,

You could accomplish this by doing a normal MaxDiff design, exporting the design into Excel, and then removing the versions that have the combinations you don't want.

I just did this in Excel in a basic version and found that there were only 9 of 300 versions without any combinations you didn't want (I used 3 concepts per screen, 12 sets per respondent).  Not as many as I was expecting, but enough to "work" with if you do it over and over a few times!  I would do the same thing with 999 versions (the max) with a few different seeds, remove "bad" versions, and combine them into a single master version.  Then re-import that design as your new design.  I would definitely test the level balance and two way frequencies of everything using the advanced design test, but it should be ok because you started with a balanced design.  Definitely test it though to be sure though!

Hope that helps!

Comments

Thanks Joel - appreciate you taking the time to answer.  I sort of assumed I would need to go down that path.  Let's say I do that with 999 versions and it generates a design with 30 "good" versions.  Would you, a) run the study with just those 30 versions (assume n=500), or b) [is this allowed?] generate two or three sets of 999 and keep maybe 100 good versions from among all of those?  All of this assuming it passes all of the necessary design tests.

---

This may be a vague answer, but it kind of depends on the advanced test with the 30 versions.  I have run studies with only 30 versions before, so I don't think that is necessarily an issue, but if certain items are not showing often (one way freqs) or combinations of items are never appearing together (two way balance) then you may need to either run more versions or do some editing to the versions (eek - be careful if you go down this path).

I just did some further testing with the 9 versions I came up with yesterday and the one way balance is perfect, but the two way frequencies aren't great.  I guess you just need to determine how far down the rabbit hole you want to go with your design.  It's a trade off of time spent optimizing vs. importance of two way balance vs. importance of your desired design style (never 3 of the same value plans appearing at the same time).  At the end of the day, there is no automatic way to do this type of detailed customization with SSIWeb unfortunately (there might be with R, not sure, but likely you don't want to learn new software just for this).

If you want my rough work and design with 9 versions that I created so far (use at your own risk), feel free to email me: joel [at] diginsights [dot] com.

---

Not vague at all.  I've got a pretty elaborate spreadsheet going of my own here so I think I'll be good.  If I might need yours I'll let you know, but thanks so much either way.