Have an idea?

Visit Sawtooth Software Feedback to share your ideas on how we can improve our products.

Interpreting HB alpha draws with multiple covariates

Hi Sawtooth community!

I'm running a ACBC study and I’m applying Bayesian tests to determine:
1) how confident I can be that attribute levels are different from one another
2) how attribute levels differ between demographic groups, and how confident I can be of those differences.

Following guidance from Becoming an Expert in Conjoint Analysis,
to test which attribute levels are different from one another, I have extracted the last 10k alpha draws and looked at for what % of draws, Attribute level A is higher than Attribute level B. If e.g. for 9500/10000 rows A is higher than B, I can say with 95% confidence that level A is preferred to B.

I understand that to test differences between demographic groups, I need to run the HB model with covariates and then look at the alpha draws. If, for example, I had gender as a covariate with female coded as 1 and male coded as 2, I would add the female adjustment to the intercept to get the female utilities. I would then look at for which percentage of rows the female utilities exceed the male utilities which are given by the intercept. My question is, how does this work if there are multiple covariates included in the model? My model includes covariates for age, education, housing tenure, income, and gender. When I add these all in to the HB model, I get a single intercept value for a reference category plus the adjusted values for each other level of each covariate. I’m a bit confused about how to interpret this.

To give an example, if say, in one row my intercept was 2, my reference category for gender is male and my reference category for housing tenure is owner occupied. Let’s also say that the adjustment for females was +1 and the adjustment for private renters was -2. If I wanted to work out the utilities for females and for private renters, would I just add 2 to both (following Bryan’s advice in this post: https://legacy.sawtoothsoftware.com/forum/13901/cbc-hb-covariates-in-the-alpha-file?show=13906#a13906 ) and then compare? If so, does this mean that the relative utilities for home owners and males can be considered to be the same? Or is it not possible to interpret the data in this way? Would it be better to run a separate model for each covariate of interest (e.g. one model where I only include gender covariates, one model where I only include age covariates etc.)?

Many thanks in advance for your help!

asked Dec 7, 2021 by Nicole (155 points)

1 Answer

+1 vote
Best answer
It's similar to the notion of a multiple regression analysis.  If the covariates may be considered mostly independent of one another (and without interaction effects among the covariates), then you can run HB  simultaneously with all covariates involved and perform the (alpha counting) tests one at a time as you previously did when there was just one covariate involved.

However, it indeed might be cleaner (and robust?) to estimate each HB model with just one covariate at a time when doing this statistical testing.  I'm not a Bayesian statistician, so I'd be glad to see somebody else in the community chime in to confirm or not this suggestion!
answered Dec 7, 2021 by Bryan Orme Platinum Sawtooth Software, Inc. (196,615 points)
selected Dec 7, 2021 by Nicole
Hi Bryan,

Thank you very much for your prompt response - makes sense to treat this in the same way as a multiple regression!

As a quick follow-up, is it possible to change which level of the covariate is used as the reference category - either in the software or through manual calculations?

Many thanks for your help!
The only way I can think of to do that is just to switch the 1s for 2s and the 2s for 1s in your demographics variable data field, then submit to the HB estimation again.