Have an idea?

Visit Sawtooth Software Feedback to share your ideas on how we can improve our products.

How many choice sets to choose?

I am doing a CBC with 36 stimuli. I can use a 12 tasks with 3 alternatives design or a 18 tasks with 2 alternatives design (apart from the no-choice option).  The D efficiency of the 12 task model = 2228,46 and the D efficiency of the 18 task model = 2720,45.

When choosing a 12 task model with 3 alternatives, I will have a loss of 18%. How do I interpretet this number and how many is 18%? I am considering to cut the length of the conjoint to avoid non ended surveys.

asked Sep 4, 2018 by RickD

1 Answer

0 votes
Strength of Design, at least from an aggregate logit perspective, is quite related to sample size.  One way of thinking about it is that losing 18% in "Strength of Design" is similar to losing 18% of your respondents.  In other words, interviewing 100 people, but really getting the equivalent of 82 people in worth toward tightening your precision.  

But, you have to consider respondent time as well, I believe.  It might take respondents more than 18% longer to complete 18 pairs versus 12 triples.

Pairs also induce slightly different choice behavior (and final utilities/attribute importances) than triples.  So, choosing between a pairs vs. triples data collection format involves more than just thinking about design efficiency and precision of betas.

I'd strongly encourage you to read a great paper that was given at the Sawtooth Software conference comparing how respondents process information in pairs vs. triples (see Meissner et al., starting on page 151):  https://www.sawtoothsoftware.com/download/techpap/2010Proceedings.pdf
answered Sep 4, 2018 by Bryan Orme Platinum Sawtooth Software, Inc. (177,015 points)
Oh, also look at Meissner et al's 2016 eye-tracking paper:  https://www.sawtoothsoftware.com/download/techpap/2016Proceedings.pdf

This paper might be even more insightful and current than the first!