Have an idea?

Visit Sawtooth Software Feedback to share your ideas on how we can improve our products.

Holdout Tasks

Dear community,

I have two identical holdout tasks in my CBC.
Within my analysis, I´ve found out that 41% of my respondents did not answer the two identical tasks in the same way.
Is this a problem or even: How to deal with these respondents? Might their choice be due to indifference or are their responses really biased? I am a bit insecure how to interpret this issue and how to maybe find out more about the issue.

Any help is appreciated.
Thanks and best regards.
asked Oct 13 by anonymous

1 Answer

0 votes
It's hard to tell anything from a single repeated choice set.  It could be that your question was particularly tough, with very nearly equally appealing concepts or it could be that your respondents just aren't paying attention and their answers are full of random noise.  

If you're interested in assessing respondent quality please check out this white paper:  https://sawtoothsoftware.com/resources/technical-papers/consistency-cutoffs-to-identify-bad-respondents-in-cbc-acbc-and-maxdiff
answered Oct 13 by Keith Chrzan Platinum Sawtooth Software, Inc. (106,150 points)
...