I searched the forum but didn't find a satisfying answer to my very basic question.

Regarding to CBC-Logit analyses: How can somebody evaluate if one model is "better" then another one?

For example:

- Model A is a model without interactions (only main effects)

- Model B is a model with interactions

Usually if you add more variables the model gets "better" in terms of the differences in Log-Likelihood value becomes bigger (compared to the nullmodel which is the same in both compared models).

I know that we now can calculate the p-Values to determine significance of the model. Also we can check the t-ratios for significance and of course we need to interpret the RLH Fit statistics.

But that all does not make sense in respect to the situation that the Log-Likelihood value simply becomes "better" (it drops closer to zero) if we add variables.

According to that, somebody could think of "as many variables (e.g. interaction) I add to the model, the better it gets", which is definitely not the true.

I also heard of "overfitting" a model, but this is not well described in the sawtooth-help and I also do know that in the end somebody has to choose a model that fits best to the questions you want to answer. But beside that I am searching for other indicators beside the content oriented to justify the model selection.

I am happy if somebody could provide me information on those questions and I do apologize if I didn't find a post that already handles this.

Many thanks!

Boris

Thank you for your helpful answer. I didnt know about this section on the sawtooth website.

May I add two follow up question? In the chapter itself, it is described how to deal with one interaction effect (price & brand). What if there are more interaction effects like also price & brand?

How much is too much? (that is what i meant in my first post with the word "overfitting".

How can one evaluate which interactions to include an which not? So for example the counts indicate that there are significant interactions between all attributes (2-way), whereas the interaction search indicates that there are interactions between Attribute A with B, C and D.

Thanks again for your reply!