Have an idea?

Visit Sawtooth Software Feedback to share your ideas on how we can improve our products.

do the characteristics of the not-chosen concepts lead to lower utility estimates for corresponding levels?

this question has been bugging me for a while and now I have a situation where the answer would have strong implications:

scenario 1:
my CBC tasks have many concepts. from self-stated questions I know that several brands or other characteristics are unacceptable. I consider to use CBC/HB to estimate a design where I remove the irrelevant concepts and only use the signal from the "relevant set" of concepts. however, if I do not inform the estimation that some levels were deemed unacceptable, would the result just assign these levels "0" impact on choice? and vice versa, if I left the concepts in and they were never chosen (I made sure of that) would the estimation recognize that these levels are actually "bad" and produce lower utils?

scenario 2:
to include the self-stated information about level preference I'd like to augment the design with tasks expressing this. there are many ways of coding this and I'm aware of the challenge of the scale factor (also potentially for scenario 1), but I'm interested in a specific case. Say we have a binary attribute (att 3 of 5) and I know that level 2 is absolutely dominant for the respondent. Using single file csv coding conventions (Resp | Task | concept | Att1 | Att2 | Att3 | Att4 | Att5 | Response) is there a difference between these two ways of coding this?:
Option A
123 1 1 0 0 0 0 0 1 ("none" is preferred over inferior level 1)
123 1 2 0 0 1 0 0 0 (the "level1" concept is not selected)
Option B
123 1 1 0 0 2 0 0 1 (level two is preferred over inferior level 1)
123 1 2 0 0 1 0 0 0 (the "level1" concept is not selected)

Option A is like an anchor task in MaxDiff and would allow me to put all sorts of rejected levels into a single augmented task. However, impact / utility is awarded to the "none" option rather than to the superior level 2 (which benefits at least not directly, or does it?).

Option B be requires to build multiple tasks if more than 1 level is acceptable. If 2 out of 5 brands are rejected then I'd need 3 tasks were each acceptable brand is chosen over the set of 2 rejected brands. So here each superior level is expressly called out / chosen. Does this lead to higher utility than in option 1?

Thanks for pointers to corresponding literature but also for brief "this is what you can expect" without detailed explanations

Cheers, Alex
asked Jul 9, 2021 by alex.wendland Bronze (2,545 points)

1 Answer

+1 vote
Best answer
When you think through the math of what MNL does, then you realize that responses of None also can provide at least some information for utility estimation for the other attributes and levels.  

Scenario 1: if you don't inform HB MNL that this respondent rejected certain concepts prior to the formulation of the relevant set, then you'll not be informing utility estimation fully about how this respondent forms preferences.  In ACBC, when respondents drop levels from consideration prior to the conjoint tasks (done via constructed lists), we add synthetic tasks to the choices for that respondent showing that the respondent "saw" these dropped levels and rejected them (how we do that is described below).

Scenario 2: There are many ways to do this.  The way we do it in ACBC for a rejected (dropped) level within an attribute is to create a new "threshold" parameter for this attribute that is coded as a 1/0 in the design matrix as a new column in the design matrix.  If you have 5 levels of brand and the respondent rejects level 3 (and we want to inform utility estimation that level 3 should be lower than levels 1, 2, 4 and 5), then we add 5 new binary choice tasks to the respondent's design matrix.  These are coded as extreme partial-profile tasks, where the other attributes are coded as zero.  Level 1 of brand is compared to the threshold concept (not the None!) where the threshold concept is coded as a 1 in the threshold column and chosen.  Same for levels 2, 4, and 5.  But, level 3 is compared to the threshold concept and rejected.  After utility estimation, we ignore (throw away) the threshold parameter associated with this attribute.  

The coding in more detail is described in a PowerPoint training we gave on ACBC the year we released ACBC.  You can write me directly if you want those slides and you know my email.
answered Jul 9, 2021 by Bryan Orme Platinum Sawtooth Software, Inc. (198,715 points)
selected Jul 9, 2021 by alex.wendland