Have an idea?

Visit Sawtooth Software Feedback to share your ideas on how we can improve our products.

number of tasks and facings for shelf display?


I am looking at doing a shelf facing display cbc for the first time and wondered if anyone had any advice on the maximum number of items and number of tasks a respondent can handle?

I have around 70 items in total and was thinking of showing 30 items per task. I was hoping for around 5 tasks per respondent but even with 6 when I used the advanced test I was getting se's of 0.06-0.07 which is above the recommended 0.05 (base of 3000 respondents).

Should I increase the number of tasks (6 already feels a little much) or increase the number of items displayed at once or live with those se's? Any advice welcome!

Many thanks

asked Jul 10, 2012 by anonymous

1 Answer

0 votes
I'll chime in with some thoughts, though I personally have little experience with these type of designs (brand and price, lots of items per task, large respondent size).  I'm writing this under the assumption that when you say 70 items, you mean something like 70 SKUs in attribute 1.

Regarding the number of items per screen, that's something that falls under the human side of research, rather than some sort of mathematical calculation.  You need to do some pretesting and make sure the task does not overwhelm someone.  There's really no right or wrong number of items, just trade-offs that you make.  More items per task results in a more efficient task, but without any additional information, I feel like 30 products on a screen might be a bit overwhelming for someone to really consider all of them, and might just be hard to fit all of those things on the screen without requiring the respondent to scroll around.

I picked up a rule of thumb from someone a lot smarter than me that it's a good idea to have levels showing in your design 5-6 times when using HB.  I don't have any math or anything to back it up, but it seems to work well.  In your case showing 30 from 70 would result in a few levels only showing up maybe once or twice, which feels a little sparse.  On the other hand, 3000 respondents is a pretty big sample size going into HB, so here's where I reference my inexperience in these types of designs and avoid speculating on just how this would turn out.  Perhaps someone else who does more of these types of studies would like to chime in.

One other note is that a typical CBC exercise usually is in the neighborhood of 12-16 tasks, and we've got a paper or two showing that people can reliably answer 20 tasks.  Again, it's a bit of a different thing when you have a lot of really simply things on the screen, but I wouldn't be afraid of going higher than 6 tasks for the respondents.
answered Jul 11, 2012 by Brian McEwan Gold Sawtooth Software, Inc. (46,970 points)
I'd comment that the paper we have stating that people can reliably do 20 tasks is from 1996.  It looked at a number of CBC data sets, done as mall-intercept or disk-by-mail.  Folks back then, under those conditions, seemed to be hanging with 20 tasks.  It's a good debate whether people in web panels who complete lots of studies per month will also be willing to give that kind of time and attention.  Some presentations at our most recent conferences have suggested that you don't get much more information out of a respondent than in the first 10 tasks.  But, so much depends.  If you have lots of SKUs to rotate through tasks, then it could require more tasks until each SKU has appeared at least a few times per respondent.  Also, the idea of minimal overlap (lack of level repeating) within each task affects how deeply we can probe respondents' preference structures.  With some level overlap in CBC tasks, then additional tasks become more revealing regarding a respondent's deeper considerations in terms of attribute and level preferences.  With minimal overlap designs, sometimes just the superficial preference information can be captured, and that can be done in 5 to 10 tasks, without much benefit in extending the respondent toward 20 tasks.