Have an idea?

Visit Sawtooth Software Feedback to share your ideas on how we can improve our products.

Constrains & Covariates in MaxDiff HB analysis


As I have described in the relevant question, that my survey is a case2 Best wost scaling and I have 6 dimensions, and each of them has 5-6 levels from the best level of the dimension to the worst level in the dimension.
I was adding a constraint that for each dimension level should be arranged logically from best to worst (for example 1>2>3>4>5>6) and this made the estimation for the model take round a day each time.
My question is what I did is correct or wrong? I mean when I estimate the HB in case 2 as I mention should I add this constraint that is mentioned above or I should not add this constraint at all and check the results without any constrains then If I found some illogical utilities (like worse levels having more utility than better levels in the same dimensions ) I should add these constrains later on??
On the other hand how could I decide which covariates are important to include n the model and which are not? (like age, gender, income...etc)

asked Dec 21, 2019 by AMYN Bronze (2,980 points)

1 Answer

+1 vote
Best answer
I usually run my unconstrained model first and then decide whether or not I need to add constraints.  I am more likely to do this in a commercial study than in an academic paper, because academics usually understand that reversals can sometimes occur.

For covariates, the consensus seems to be that if you have really good covariates, ones that are strongly related to differences in choice behavior, that those can be valuable to include in HB models.  More general variables, like the general demographics you describe, haven't often been found to improve on the ability HB already has to find heterogeneity, so we typically do not include them.  I don't have an easy suggestion if you do try to include them but to run the model with and without a constraint and then see if the utilities are more significantly different with the covariate than without it - but this is an effort that sounds like it is going to take you a really long time, and I suspect you'll find that it doesn't have value commensurate with the time you'll be investing.
answered Dec 21, 2019 by Keith Chrzan Platinum Sawtooth Software, Inc. (117,375 points)
selected Dec 21, 2019 by AMYN
I have added four covariates (3 no/yes & 1 ranking of the six dimensions), the RLH improved by 0.025-0.03, do you believe this is a significant improvement that supports adding these covariates or should I try removing /adding one at a time and rerun the models to see the least number that have the most significant improvement?
I would look at them one at a time to see which ones(s) are doing the work.  I'm not aware of a formal test that determines what size of RLH improvement is significant for the addition of how many covariate degrees of freedom.