Have an idea?

Visit Sawtooth Software Feedback to share your ideas on how we can improve our products.

Minimum amount of Choice Tasks in CBC, HB


I read somewhere that you need like a minimum of 10 Random Tasks to do a good job with individual level prediction. Thats important to me, because I want to do a segmentation later. Is there a formula to calculate the number of choice tasks? Or a reliable source?
I have 6 attributes, each with 3 levels. And I have 3 concepts per task (excluding none).
I would appreciate any help!
asked Jul 28 by sullivan (140 points)

1 Answer

0 votes
We conducted a CBC/HB stress test a couple of years ago and found strongly diminishing returns on additional questions after about 10 or 15, even with robotic respondents who don't get fatigued.  Make those respondents humans who can get bored or tired and the story doesn't get better for larger numbers of questions.  

We posted this paper to our LinkedIn discussion group, but if you send me your email I'll be happy to email it to you:  keith@sawtoothsoftware.com
answered Jul 28 by Keith Chrzan Platinum Sawtooth Software, Inc. (93,025 points)
Thank you for your quick reply! Hm thats a problem to me! I want to do two CBC in one survey, one for a high involvement product and one for a low involvement product. I know that too many questions lead to worse quality, but I also need a minimum of questions to get good results for my HB. Therefore I wanted to use 10 random tasks for each product, making 20 tasks in total (excluding 2 hold outs). Unless there is a source, which says I can use less than 10 random tasks for a HB analysis.
You may be able to get by with a few fewer than 10 - I show results for smaller numbers in the paper I mentioned.
Thanks! I sent you an Email with sebastian.sxxx@xxx.de
I've not received your email, but here's a link to the article:  https://sawtoothsoftware.com/sawtoothsoftware/assets/cj1rjn27k88ow8c0
Thank you very much! Strange that you did not receive my email.
So if I get your paper right, the number of questions depend on the correlation/accuracy I want to archieve. Is there a rule of thumb how much percent correlation is necessary? Further you suggest that I need atleast 8 questions and maximum 20 questions, right?
Yes, correlation improves as the number of questions increases, but with diminishing marginal returns.  Quality rises more slowly with each additional question and likely more slowly still with human respondents.  There is no rule of thumb about how high a correlation you need.  I would not go lower than 8-10 questions and the evidence from human respondents suggests that with more than 20 questions your results may start to get worse because of respondent fatigue.
Thank you very much!