Have an idea?

Visit Sawtooth Software Feedback to share your ideas on how we can improve our products.

CBC Model fit

Dear all,

for the logit model you typically evaluate the Chi-Square statistic (2*(LL1-LL0) for significance.

Do you typically do this step also for a HB model? Or only RLH and Pct. Certainty?

asked Aug 31, 2021 by anonymous

1 Answer

0 votes
Log-likelihood tests (for Sawtooth Software tools) are associated with Aggregate logit and Latent class logit estimation.  With HB, we tend to look at RLH and Pct. Certainty (also known as McFadden's Pseudo Rsq).

With RLH and Pct. Certainty, be careful, because striving for the highest fit possible under HB isn't necessarily the goal that leads to better predictions of held out (new) observations.

For example, one can easily increase RLH by re-running the HB model with a higher prior variance setting.  But, this can lead to overfitting at the individual level.
answered Aug 31, 2021 by Bryan Orme Platinum Sawtooth Software, Inc. (198,715 points)