Express MaxDiff design lacks of balance principles

Question

Hi!

I was trying to run a Express MaxDiff over 35 items, as we are asking the users to go through two exercises (one asking most/least unique, the other one most/least attractive). So we set up the design in such way that every respondent will evaluate 20 items of those 35, showing 5 items per task and going through the tasks 12 times.

So I created both files and combined them but my final design lacks all of the principles, starting from frequency balance, as there are shown just 593 times vs other shown 637 times. We don't have either position balance nor orthogonality.

Is this common, how will this affect the results?

Answer 1

Atenea,

That sounds like a strange result.  How did you choose your subsets of 20 items  - did you use a design to do that or did you use constructed lists?  Either way I think that's where you may find your problem.

On another note, why Express MaxDiff?  Repeatedly we've seen that Sparse MaxDiff will give you more valid utilities than Express.  You might want to think about going that route.  With 35 items you could show each item twice, and show 14 tasks with 5 items each.

Comments

I actually subset using the Sawtooth MaxDiff Tool, having 35 items showing 20  items per task having one task, to get a balanced design. Should I be using other tools?

Is there any article related on the efficiency of the Sparse MaxDiff vs the Express?

---

Atena,

I'm not entirely clear on what you're doing, but it's clearly not working if you're getting an imbalance.  you might contact our support folks (support@sawtoothsoftware.com).  As far as Sparse vs Express, see the Serpetti et all paper in our 2016 Conference Proceedings, here (https://sawtoothsoftware.com/support/technical-papers/100-support/proceedings/1776-proceedings2016) or see this white paper (http://www.sawtoothsoftware.com/support/technical-papers/175-support/technical-papers/maxdiff-best-worst-scaling/1493-a-parameter-recovery-experiment-for-two-methods-of-maxdiff-with-many-items).  We followed that white paper up with a paper we'll be showing at our upcoming Turbo Choice Modeling event, and I can get you a copy if you like, just write me at keith@sawtoothsoftware.com.