Have an idea?

Visit Sawtooth Software Feedback to share your ideas on how we can improve our products.

Change of holdout task after launch?


can we change the level of a holdout task in a cbc after the survey has started? So far there are only a few respondents who completed and we would like to adapt the holdout task we included as dominance test. Could there be problems later downloading all data using lighthouse?

Kind regards,
asked May 28, 2020 by Andrew Bronze (1,255 points)

1 Answer

0 votes
Fixed tasks aren't actually part of the design the software generates, though they will be present if you export your design from the software.  When you run analysis, the answers will basically be wrong for the respondents you already collected and the software will warn you accordingly when you change the fixed task, but it won't provide any warnings once the data is collected and you won't really know that respondents 1-10 actually saw something different.
answered May 28, 2020 by Brian McEwan Platinum Sawtooth Software, Inc. (53,260 points)
Thank you, Brian, for the fast response.
So we can change the holdout levels, upload the survey again and nothing bad will happen. But we keep in mind that the first few respondents saw a different holdout task. Thanks for the advice.
Seems to be the case.  I did little testing with a sample file, though I'm always hesitant to give a 100% stamp of approval there aren't any unforeseen circumstances.  I'm pretty sure you're fine, with the main problem just being from a data perspective, there will be no way to know that respondent 19 saw the old holdout and beginning with respondent 20, they see the new one.