Have an idea?

Visit Sawtooth Software Feedback to share your ideas on how we can improve our products.

Sample size calculation and blocking


in research papers, I keep coming across Orme's rule of thumb for calculating the sample size for a CBC. I would like to know what role does the number of questionnaire versions play in using the formula? Does it only apply when using one questionnaire version?

As an example, with 75 respondents, 10 tasks, 3 concepts, 3 levels, the formula yields 750 level appearances and is below the recommended of 1000 times. Increasing the tasks to 20 (and blocked into 2 questionnaire versions) would yield 1500. The reason I ask is because we have a realtively small sample (n=75) in a current project. However, taking blocking into account in the formula would thus easily yield the recommended number of appearances even with a small sample.

Or what am I seeing wrong?

Kind regards,
asked Jan 21 by Andrew Bronze (1,265 points)
edited Jan 21 by Andrew

1 Answer

0 votes
Best answer
Andrew, the number of blocks doesn't enter into the rule of thumb at all - it's not needed for the calculation.  The formula applies as well to a design in 3 or 30 or 300 blocks.  

So yes, your sample size is a little slim and if you could increase it you should.  That said, we've often seen that CBC works well even with smaller sample sizes.  What you might want to do is run the design test that's built into the software, and take a look at the standard errors that result for your attribute levels.  If you can live with those standard errors, then I think you're OK to proceed, even if you're below the rule-of-thumb for sample size.
answered Jan 23 by Keith Chrzan Platinum Sawtooth Software, Inc. (110,575 points)
selected Jan 23 by Andrew
Dear Keith, thank you very much for the quick reply.

And yes, we'll check standard errors. Perhaps switching to an ACBC would be more appropriate than a CBC with such a small sample.