Have an idea?

Visit Sawtooth Software Feedback to share your ideas on how we can improve our products.

Is 2x5 levels and 1x7 levels already problematic regarding the number of levels effect in a CBC?

I want to include 3 attributes into my CBC.
2 of them have 5 levels each. But for the third one (price) I would like to include 7 levels because I want to provide a broad range of price to choose from and I don't want to have too large gaps between the different prices.

I already read about the number of levels effect in the Technical Papers of Sawtooth and I understand that the levels for all attributes should be as equal as possible. But I am not sure if 2x5 & 1x7 levels is already a problem or if that still works.

Maybe someone could give me advice or already tested something similar and can share his/her experiences.
asked Jul 22, 2019 by Mary (340 points)

1 Answer

0 votes
The number of levels effect for quantitative attributes has been demonstrated to be problematic when you compare 2 levels of an attribute to 4 levels of the same attribute.  Academics demonstrated that you can conduct the same conjoint analysis experiment where in one group of respondents a quantitative attribute gets 2 levels, but for a different group of respondents that same attribute is shown in 4 levels (but where levels 1 and 2 from the first group and levels 1 and 4 from the second experiment are the same).  In that case, the importance of that attribute expands by about 30 to 60% when created as a 4-level attribute.

This finding got some academics some published papers in the 80s.  However, a researcher from SKIM group (Marco Hoogerbrugge; https://www.sawtoothsoftware.com/download/techpap/2000Proceedings.pdf) tested the number of levels effect in 2000 when comparing quantitative attributes with more reasonable numbers of levels, such as 3 vs. 5 levels, or 5 levels versus 9 levels, etc.  He found that the number of levels effect was much reduced when comparing two experiments with more levels, such as 3 levels versus 5 levels.  

Here's a sentence from his conclusions: "The conclusion is, the NOL effect clearly exists and is large when comparing 2-level attributes with more-level attributes, but the effect is questionable when you only have attributes with at least 3 levels. "

So, it seems that the number of levels effect is particularly manifest when comparing the idea of using 2 levels to a larger number of levels.  But, hardly any researcher would think about representing an attribute with just 2 levels, so this seems like a corner case.

I doubt the number of levels effect would be very big when comparing a 5-level to a 7-level attribute.  That said, if it's easy to do, I would try to make quantitative attributes have the same number of levels if I had the flexibility.
answered Jul 22, 2019 by Bryan Orme Platinum Sawtooth Software, Inc. (175,715 points)