The main difference between separate HB-runs & HB-runs with covariates

Question

Hi,

I am slightly confused as to how running separate HB-runs based on 5 segments for example, and running HB with covariates defined by the same variable differ?

Don't both make individual respondents "borrow" information principally from the means of their defined group rather than from the overall population means?

Answer 1

Lise,

I had the same question, so I fielded an R&D study to find out.  I reported this on our LinkedIn discussion group a couple of years ago, but the bottom line is this:  I didn't find significant differences in how well the model fit holdout data (as measured by mean absolute errors of prediction).  

When I presented this work at one of our conferences, I found the audience about split on how they handled estimation in cases like this, and that they seemed to split more on which method they found more convenient than for any theoretical reasons.  

So I think you're safe to proceed as you like.

Comments

Thanks for a fast reply!

Answer 2

In a run with covariates, there is still a set of means for the entire population, as well as for each subgroup.  An individual's betas will be influenced by the means for their subgroup and the overall population.

However, the largest difference from a computing standpoint will be due to the fact that HB is highly dependent on random values generated from normal distributions.  When you change the set of respondents, you change the entire course of values generated which will change the results regardless of whether covariates were used or not.

Comments

Thanks for a fast reply!