Have an idea?

Visit Sawtooth Software Feedback to share your ideas on how we can improve our products.

Number of covariates and their influence on significance

Hello, everyone,

I did a study using Lighthouse Studio, but I evaluated it with R and the 'Apollo' package by Stephane Hess and David Palma. Whereas in Lighthouse Studio the covariates affect all part-worth utilities, 'Apollo' allows you to differentiate which attribute the individual covariates should affect.

I learned from the current scientific literature that only a few covariates with as much explanatory information as possible should be used. Now the following questions arise:

a) What happens if I add more covariates to the model? So far I have understood that in extreme cases the effects of the covariates are zero, i.e. not significant. However, can it happen that covariate A alone is significant, B alone is significant, but A and B together are not significant? In regression analyses, I know the problem of multicollinearity. Nevertheless, the HB model is different so I am not sure if I am on the safe side, even if A and B do not correlate.

b) Does the sample size have an influence on the number of covariates recommended?

I look forward to your answers.
asked May 8, 2020 by Nico Bronze (1,160 points)
Can't anyone say anything about it?

1 Answer

0 votes
This is an area I haven't experimented with a lot regarding sensitivity to multicolinearity and overfitting.  I've heard sometimes conflicting advice from HB experts over the years regarding this area.

Entering two covariates that were perfectly correlated seems to me to be a bad thing, as you expect.

Too many covariates not only slows down the estimation, but could lead to overfitting.

More sample size should help reduce the likelihood of overfitting, so larger sample sizes could justify using more covariates than smaller sample size.
answered May 19, 2020 by Bryan Orme Platinum Sawtooth Software, Inc. (201,765 points)
Thank you Bryan for your answer.