Compare models with different combinations of covariates in CBC/HB

Question

From reading section 4.10.3 of the manual and 'Applications of Covariates within Sawtooth..', I am comfortable with determining whether a covariate is significant or not. However, I don't understand how to compare analysis with one covariate (ex: gender) to an analysis with multiple covariates (ex: age and education) to determine which model is a better fit as a whole.

Is there an r^2_adj equivalent to the Fit RLH statistic or a chi-sq test to compare models for HB?

Answer 1

You can do this via the Bayes Factor ( Kass and Raftery 1995; "Bayes Factor"), which is calculated as B_10= 2*|LMD1-LMD2|.
 LMD ist the Log marginal density of the model, it's the same as the log likelihood value.
 If the bayes factor B_10 is greater than 3.2 you have substantial evidence, that the model is better, if the value is greater than 10 you have strong evidance, greater than 100 means decisive evidence... You can find this information the paper as well.

Here you find the Kass and Raftery paper.: https://www.stat.washington.edu/raftery/Research/PDF/kass1995.pdf