Have an idea?

Visit Sawtooth Software Feedback to share your ideas on how we can improve our products.

Compare models with different combinations of covariates in CBC/HB

From reading section 4.10.3 of the manual and 'Applications of Covariates within Sawtooth..', I am comfortable with determining whether a covariate is significant or not. However, I don't understand how to compare analysis with one covariate (ex: gender) to an analysis with multiple covariates (ex: age and education) to determine which model is a better fit as a whole.

Is there an r^2_adj equivalent to the Fit RLH statistic or a chi-sq test to compare models for HB?
asked Jul 12, 2018 by axn9077 (160 points)

1 Answer

+1 vote
Best answer
You can do this via the Bayes Factor ( Kass and Raftery 1995; "Bayes Factor"), which is calculated as B_10= 2*|LMD1-LMD2|.
 LMD ist the Log marginal density of the model, it's the same as the log likelihood value.
 If the bayes factor B_10 is greater than 3.2 you have substantial evidence, that the model is better, if the value is greater than 10 you have strong evidance, greater than 100 means decisive evidence... You can find this information the paper as well.

Here you find the Kass and Raftery paper.: https://www.stat.washington.edu/raftery/Research/PDF/kass1995.pdf
answered Aug 19, 2019 by Andrea
selected Aug 19, 2019 by axn9077