Have an idea?

Visit Sawtooth Software Feedback to share your ideas on how we can improve our products.

When to include interaction effects in HB analysis

My counting analysis revealed that 4 of my 10 two-way interactions are significant:
1) χ 2 (DF=12)=24.89 p<.05
2) χ 2 (DF=12)=28.64 p<.01
3) χ 2 (DF=4)=31.14 p<.01
4) χ 2 (DF=12)=29.18 p<.01

How do I decide whether or not to include these in my HB analysis?

Thank you very much in advance!
asked Feb 6, 2020 by anonymous

1 Answer

+2 votes
Aggregate (pooled) analysis such as counts or aggregate logit can often indicate statistically significant interaction effects.  However, most of the time these interaction effects are due to unrecognized (unmodelled) heterogeneity in the data.  Once you estimate individual-level utilities via something like HB, we have found that most of the interaction effects that appear significant in aggregate analysis fade (are explained) away and are no longer needed.

For more evidence regarding that finding, including how to test this on your own for your CBC dataset using the "CBC/HB Model Explorer", see this white paper:

And, the part dealing with interaction effects in HB begins on page 10.

The CBC/HB Model Explorer works if you have the CBC/HB Standalone System installed first.  

CBC/HB standalone system can be installed from: https://www.sawtoothsoftware.com/support/downloads/download-cbc-hb

The CBC/HB Model Explorer can be installed (including its documentation) from: https://www.sawtoothsoftware.com/support/downloads/tools-scripts

The CBC/HB Model Explorer works by randomly selecting some of the respondents' choice tasks for holdout validation and estimating the utilities using the remaining choice tasks.  Models both with and without interaction effects are tried, to see if the inclusion of interaction effects appears to improve the hit rate for the validation (holdout) choice tasks.
answered Feb 6, 2020 by Bryan Orme Platinum Sawtooth Software, Inc. (184,140 points)