Have an idea?

Visit Sawtooth Software Feedback to share your ideas on how we can improve our products.

The main difference between separate HB-runs & HB-runs with covariates


I am slightly confused as to how running separate HB-runs based on 5 segments for example, and running HB with covariates defined by the same variable differ?

Don't both make individual respondents "borrow" information principally from the means of their defined group rather than from the overall population means?
asked Jan 2, 2019 by Lise Kristiansen

2 Answers

+1 vote

I had the same question, so I fielded an R&D study to find out.  I reported this on our LinkedIn discussion group a couple of years ago, but the bottom line is this:  I didn't find significant differences in how well the model fit holdout data (as measured by mean absolute errors of prediction).  

When I presented this work at one of our conferences, I found the audience about split on how they handled estimation in cases like this, and that they seemed to split more on which method they found more convenient than for any theoretical reasons.  

So I think you're safe to proceed as you like.
answered Jan 2, 2019 by Keith Chrzan Platinum Sawtooth Software, Inc. (117,375 points)
Thanks for a fast reply!
+1 vote
In a run with covariates, there is still a set of means for the entire population, as well as for each subgroup.  An individual's betas will be influenced by the means for their subgroup and the overall population.

However, the largest difference from a computing standpoint will be due to the fact that HB is highly dependent on random values generated from normal distributions.  When you change the set of respondents, you change the entire course of values generated which will change the results regardless of whether covariates were used or not.
answered Jan 2, 2019 by Walter Williams Gold Sawtooth Software, Inc. (23,405 points)
Thanks for a fast reply!