Have an idea?

Visit Sawtooth Software Feedback to share your ideas on how we can improve our products.

Relevance of covariates in HB estimation

I have a question regarding the usefulness of covariates in my ACBC HB estimation. Here's an interesting excerpt from the CBC/HB manual:

"If the percent of draws (associated with a part-worth utility) that have the same sign is 95% or greater, this is often taken as evidence that this realization of the covariate has a significant effect (90% confidence level, two-tailed test) on the part-worth utility estimate.  If a relatively large number of columns for a covariate have significant weights, then this gives evidence that the covariate is useful."

I have a couple of covariate candidates, but it's tough to judge whether they truly make a difference or not. How can I decide whether or not to include a particular covariate? If e.g. 5 out of 18 columns are significant (> 95% same sign), is that unrelevant? What could be a threshold here to make a decision?

Thank you!
asked Apr 28, 2021 by danny Bronze (1,310 points)
Also I'm a bit confused that the intercept column represents the last level of an attribute. I would rather expect to have utilities without any covariate influence as the "base" / intercept, so that I can compare those with the covariate values.

1 Answer

0 votes
Tough question that's usually a judgement call.  The good news is that the quality of your results and predictions is fairly robust even in the face of using random or spurious covariates.  

When researchers get serious about whether to include covariates or not, they use judgement regarding whether the covariate is correlated with respondent preferences.  And, if they're able, they see whether including the covariate improves prediction of held out data--preferably out-of-sample held out choice task data.
answered Apr 28, 2021 by Bryan Orme Platinum Sawtooth Software, Inc. (201,565 points)