Have an idea?

Visit Sawtooth Software Feedback to share your ideas on how we can improve our products.

Very poor chi-square in Test-Design-Report CBC


I am preparing for starting the pre-test phase  of my study right now and wanted to test my design beforehand. The standard errors for each attribute are below .05, but the chi-square value is very poor.

I have 15 degrees of freedom (20 attribute levels - 5 attributes) and my chi-square of the report is 5.66608 although in my understanding it should be at least 25 if I want it to be significant (p<.05).

 Iter    5  Log-likelihood = -4779.88251  Chi Sq = 5.66608  RLH = 0.25021

The RLH is also really weak if you consider that I have 4 concepts per choice task.

In one of the Sawtooth videos it is said that one should only make sure it says "*Converged" below those values. Is this the only thing I have to consider or is my design deficient?
asked Jan 17, 2018 by Carla Merz (230 points)

1 Answer

0 votes
The robotic respondents are answering randomly, so you expect a really terrible fit statistic and really terrible Chi-Square.  Even though the test robotic respondents are random, the experimental design is sufficient to calculate standard errors that should be quite similar to what you will see when you obtain real respondents.
answered Jan 17, 2018 by Bryan Orme Platinum Sawtooth Software, Inc. (177,015 points)