Have an idea?

Visit Sawtooth Software Feedback to share your ideas on how we can improve our products.

Influence of mixed approach vs. attribute balance on number-of-levels effect

Is there any evidence on how mixed approach (BYO Modification Strategy) influences number-of-levels effect in ACBC studies? It seems logical to me that the mixed approach enforces number-of-levels effect (behavioral effect) since attributes with more levels are modified more often in comparison to "attribute balance".

Are there any studies concerning that issue?
asked Apr 16, 2018 by Tim Geyer (270 points)

1 Answer

0 votes
That's a good question.  I don't think any research has ever been published on this.  Be aware, however, that the number of levels effect is strongest when comparing 2 levels to 4 levels of a quantitative attribute (where the extreme levels are the same).  Doubling from 3 levels to 6 levels, the number of levels effect is less, etc.
answered Apr 16, 2018 by Bryan Orme Platinum Sawtooth Software, Inc. (176,815 points)
Why is the effect strongest when comparing 2 levels to 4 levels of a quantitative attribute (assuming you have price in mind)? Was there research published on that matter?
Back in 2014, I made a post on this thread about the Number of Levels Effect.  I'll repeat it below:


The NOL effect is an interesting one that has caused many a researcher to worry.  The late Dick Wittink is credited with authorship or co-authorship on the early NOL effect papers.  Every experiment I'm aware of that he and his co-authors did examined the case of using one group of respondents who saw just two levels of a quantitative attribute (such as price) vs. four levels (while holding the range constant), such as:

Respondent Group 1:

Respondent Group 2:

When he compared the importance score for the manipulated attribute from Group 1 to Group 2, (if memory serves) he found about a 1.5x increase in importance.

But, Marco Hoogerbrugge in the 2000 Sawtooth Software Conference Proceedings (http://www.sawtoothsoftware.com/download/techpap/2000Proceedings.pdf) looked at the number of levels effect when moving from:

3 to 5 attribute levels
5 to 9 attribute levels
5 to 8 attribute levels

He concluded that "...the NOL effect clearly exists and is large when comparing 2-level attribute with more-level attributes, but the effect is questionable when you only have attributes with at least 3 levels."

It is striking to note that hardly any market researcher would ever use just 2 levels of a quantitative attribute such as speed, price, or quantity.  So, if Marco's conclusions are right, then the NOL effect is less a trouble than the famous papers that highlighted it would lead us to believe.  Usually we're asked by the client to use 4 levels of speed and 6 levels of price, etc.  We're hardly ever asked by the client to use 2 levels of speed and 6 levels of price.
And, number of levels effect I think has only been shown for quantitative attributes like price, speed, and weight.  Other categorical attributes (like brand, or presence/absence of a feature) the researcher often just is given the number of levels such as that is observed in the real world too.  

It seems that a major part of the observed number of levels effect in conjoint analysis (as seen when testing 2 levels of a quantitative attribute vs. 4) is psychological.  So, if the number of levels effect applies to categorical attributes like brand and style, then that psychological effect also probably exists for real choices in real stores.  For example, if the number of brands is particularly large on the grocery store shelf, then it invites the buyer to pay a relatively increased amount of attention to brand vs. other features.  So, perhaps conjoint analysis should just mimic the real world presentation of products as much as it it can and that it makes sense to do.
My study includes categorical attributes (2-4 levels each) only, except for one attribute (response time). However I followed your suggestion to impose individual constraints due to a tiny sample size. Do constraints make categorical attributes vulnerable to NOL effect?
In cases of using a great many levels of an attribute with full-ranking constraints (such as 20 or more levels), then chaining full-ranking constraints within HB analysis can indeed build a number of levels bias argument.  I would not worry at all about constraints leading to enhanced number of levels effect if your categorical attributes are from 2 to 4 levels each.