MaxDiff Analysis from external data export

Question

Hi everybody,

our client provided us the MaxDiff results of our market survey as raw data (Collected with the market survey program Alida). It is an excel file with one line per respondent and per screen seen (we had 13 screens per respondent and thus have 13 lines per respondent).

In each line we have the information whether a feature has been seen, has not been seen, has been selected most attractive or has been selected least attractive. That is the structure of our raw data - and we have two questions regarding this:

1. Is it possible at all and if yes, how to calculate the relative MaxDiff scores per feature across all respondents in excel directly?

2. If the first approach is not feasible, is it possible to upload such a data structure in the sawtooth program to calculate those scores there?

Thank you a lot in advance!
Best
Adrian

Answer 1

Yes, you can format the data for HB estimation using the "Single CSV Format", as described in our CBC/HB software manual.  You need to be licensed to use our CBC software to do this, then you install our CBC/HB Standalone estimation module.

The documentation for Single CSV Format can be found in the CBC/HB Manual:  https://content.sawtoothsoftware.com/assets/276545e9-0445-474c-b01c-f5b24c3eba6d

Search for the phrase "Single CSV Format", and there's a section (page 25) describing how to code best-worst CBC experiments.  The trick is that a MaxDiff study looks just like a CBC for these purposes, except it has just one attribute with many levels.

Next, your question about computing MaxDiff scores directly in Excel.  If you want to do just aggregate (population-based) analysis, not individual-level analysis...and if your experimental design was perfectly balanced and orthogonal, then you could use a simple counts methodology that would be 0.99 correlated with results from an aggregate MNL (logit) estimation.  

Simply compute %best for each item and % worst.  The MaxDiff score is %best-%worst.  Again, your design needs to be perfectly balanced and orthogonal for this to provide unbiased estimates.

Warning: don't try this $best-%worst approach for individual-level score estimation unless the design is perfectly balanced and orthogonal at the individual level and unless each item was seen at least 4x per respondent.  Without enough data at the individual level, you'll risk overfitting the data with the simple counts-based individual-level approach.

Comments

Hey Bryan,

thank you so much for your very quick answer, very helpful!

I have one follow-up question.
We are now talking about computing scores scaled between -1 to 1. In previous projects we used scores that varied between 0 and 100 with the sum of all features being 100 (probability scaled). Is there also possible way to compute those scores with the data described in my initial post?

Thank you a lot in advance.
Best

---

In your initial post, you were asking about estimating scores using Excel.  But, you asked about uploading the raw MaxDiff data to a Sawtooth Software routine as well.

So, which route do you intend to go?  

Do you intend to estimate the scores at the individual level or just at the pooled aggregate level?

Do you want the scores to be based on the raw (interval, logit-scaled) scale, or based on the probability (exponential transformed) scale?

---

Best case would be if it is possible in Excel directly.

We are looking for pooled aggregate level probability scale scores.

---

If your design is perfectly balanced and orthogonal, then simply computing %best-%worst will be correlated 0.99 or better with scores from aggregate MNL.

To be clear, that means each item needs to appear an equal time across respondents.  And, each item needs to appear with every other item an equal number of times across respondents.

Failing perfect balance and orthogonality in the experiment, the %best-%worst will have some degree of bias in the parameters.  If the design is very close to perfect, but not exactly perfect, the bias will likely be small when using the simple counts approach I suggested.

---

Thanks for your reply!

Fortunately our design is orthogonal.
And totally understood how to compute those scores now.

Still, is it also possible to convert those scores (lets say I now have 20 scores between -1 and 1) in a scale between 0-100?

Our clients are used to that scale, unfortunately.

---

After computing the scores using %best-%worst, just find the multiplier and the intercept shift that linearly transforms the scores to run from -100 to +100.

---

Sorry, I was a bit imprecise.

What I mean is that the lowest value of an item could be 0. The highest one could be 100. But at the same time the sum of all values is 100.

In that way you could read it like that: e.g. Item 3 (Score of 16.4) is twice attractive as the Item 8 (Score of 8.2). And at the same time you do not have item attractiveness below the score of 0.