Have an idea?

Visit Sawtooth Software Feedback to share your ideas on how we can improve our products.

Why does Sawtooth show me an efficiency warning?

I keep getting the following warning when I run the efficiency test in Sawtooth:

**Warning: We strongly encourage you to further investigate the efficiency of your design prior to fielding this study.

However, I have taken steps to improve the efficiency of the design and the standard errors are well below the recommended .05 for main effects and .1 for interaction effects.

Is there something I am missing? Or does the warning just show up anytime you include a prohibition?
asked Jul 10, 2018 by anonymous

1 Answer

0 votes
This message is displayed if there are prohibitions, or if the number of versions * number of tasks is less than 100.
answered Jul 10, 2018 by Walter Williams Gold Sawtooth Software, Inc. (20,130 points)
So, it's just calling your attention to the fact that your design has some aspects that are not ideal (either a prohibition or <100 total versions*tasks)...so you should make sure you think carefully about how suitable it is prior to fielding the study.  

If the main effects for your planned sample size are all 0.05 or lower, then in our experience that's a sign that things will probably go well for estimation of main effects.  If you have a prohibition between two attributes, you will not be able to estimate the interaction term between those two attributes...but you may be able to estimate interaction terms between other attributes.  You say that the standard errors for the interaction effects you're interested in are 0.1 or less for the planned sample size.  That also suggests things will probably go well.
...