statistical tests & group comparison

Question

Dear Bryan,
@esearchers typically conduct statistical tests to assess whether one level within an attribute has a higher or lower effect on choice than another level within the same attribute
Q: What kind of statistical tests are there? Just looking which level has the highest part-worhs? If there are  other tests could you  refer to some paper or subject?

@they compare groups of people to assess whether one group of people thinks that a given level is more preferred (or an attribute is more important) than another group does.

Q: How is this asessed? Looking at part worths, ldoing t-tests or a smoother way?
Thanks in advance,
markus

Answer 1

There are Frequentist and Bayesian tests that can be done for the sample, to test whether one level within an attribute is higher than another.

For the Frequentist tests, one typically uses the normalized (zero-centered diffs) individual-level output.  One runs a matched pairs t-test.

For the Bayesian tests, one typically examines the "alpha draws" file which contains the estimates of the population mean utilities across iterations of the HB algorithm.  One typically ignores the first 5K or 10K draws focuses only on the draws after that (after convergence is assumed).  Counting the draws to observe what % of the draws indicate that one level of an attribute is higher than another level within the same attribute is a direct assessment of the confidence level.

When comparing groups of respondents on a specific attribute level, then the Frequentist approach involves using the Zero-Centered Diffs utilities and running an F-test.  

For Bayesian tests comparing groups of respondents, one typically uses HB's option of using the group variable as a covariate.  The "alpha draws" now become more complicated in the output, but may be referenced for counting how many of the used draws show one group preferring that level over a different group.