Have an idea?

Visit Sawtooth Software Feedback to share your ideas on how we can improve our products.

Unaccepable attribute level in CBC


I did a CBC analysis with a choice set consisting of two product options and one none option. There are four attributes included with two levels each. I would like to find out whether one attribute level is unacceptable. Do you know how I can test for an unacceptable level?

Thank you very much,

asked Feb 4, 2014 by Caroline

2 Answers

+1 vote
If you make some assumptions about what characterizes a respondent who believes a level is unacceptable, then you might be able to do some sort of analysis.  For example, you might make the assumption that across all the tasks in your study, if the respondent never picked a product concept containing a certain level, then it was unacceptable.  This would involve exporting the CBC data to a .csv file out of SSI Web and doing some additional analysis in Excel or your favorite software package for analyzing .csv files.

However, this approach has its weaknesses, including:

1.  Maybe over the limited number of choice tasks in your study, it was purely chance (due to the experimental design received by this respondent) that the respondent seemed to avoid a particular level and never picked a concept including it.

2.  Maybe a respondent truly did think a level was unacceptable, but due to a careless error accidentally picked a concept with that level.

3.  Maybe some respondents answered the first few choice tasks well, but then became fatigued and gave some random answers in the end, thus negating your ability to detect that they consistently felt a level was unacceptable.

Our ACBC software provides a way to try to get at unacceptables: observing over a series of choice tasks if a respondent consistently is avoiding a particular level...and then actually prompting the respondent on the screen to confirm or deny that the respondent thinks that level is unacceptable.  While this seems like a plausible approach to very accurately identify unacceptable levels, a recent paper by Kevin Lattery found that even under ACBC's approach of identifying unacceptable levels, respondents would choose supposedly unacceptable levels later in holdout choice questions within the same survey.  We don't know for certain whether the errors are in the holdouts or in the ACBC survey, but it points out the challenges of assessing unacceptables in conjoint questionnaires.

Academics such as Allenby and Gilbride came out with ways via HB analysis to try to infer unacceptable levels, but to my knowledge no software has been published to do what they proposed.
answered Feb 4, 2014 by Bryan Orme Platinum Sawtooth Software, Inc. (176,815 points)
Thank you very much for your detailed answer! I appreciate it!
0 votes

Bryan's suggestion is a good way to go to identify attribute levels which a respondent finds aversive in the course of their survey.  Of course the more CBC questions you have, the more opportunity you give respondents to avoid an attribute level and the more sure you can be that they are avoiding it (i.e. avoiding a level 20 times in 20 questions is more definitive than avoiding it 6 times in 6 questions).

That said, there has been some criticism over the years about the concept of unacceptable levels in conjoint analysis.  "Unacceptable" is such a strong word, implying a virtually infinite penalty associated with a level.  Academic Joffre Swait did some interesting work on this in the late 1990s using a "soft penalty" model where he quantified the penalty for attribute levels which respondents claimed were unacceptable.  What he found was that the penalties associated with unacceptable levels was usually finite and could often be outweighed by differences in other attributes.  For example, I go shopping for a new car and say I'll not pay more than $30,000 ($30,000 is unacceptable) but then I go to the showroom and find that $32,750 buys the car I want with so many bells and whistles that I buy it anyway.
answered Feb 4, 2014 by Keith Chrzan Platinum Sawtooth Software, Inc. (95,675 points)
Thanks a lot! I'm going to take a look at the paper!