Have an idea?

Visit Sawtooth Software Feedback to share your ideas on how we can improve our products.

t-test instead of f-test

Hi everyone,

in the statistical testing chapter of the book (Getting started with conjoint analysis) it says:
"For comparing groups of people on the part-worth utilities, the f-test Is commonly used. For comparing levels within attributes, we recommend matched sample t-tests."

Why exactly is that the case? F-tests compare the differences in variances while t-tests compare the differences in the mean. So don't they measure different things?        

Kind regards
asked Jul 31 by maxive94 Bronze (1,410 points)

1 Answer

0 votes
The t-test is how you test for differences between exactly two groups.  

The F test (analysis of variance or ANOVA) is the preferred way of testing for differences in means when you have 2+ groups because if you wanted to test differences among respondents from 4 countries, say, it would involve 6 t-tests and therefore an increase of experimentswise error and an increase in false significance in your tests.  The omnibus test provided by ANOVA controls against this experimentwise error.
answered Jul 31 by Keith Chrzan Platinum Sawtooth Software, Inc. (114,400 points)
Thank you! Is there any paper that deals specifically with stat testing for conjoint analysis? The recommendation in the book chapter involves mainly f-tests and matched sample t-tests. It seems a bit basic. I figured I need other tests like the Kruskal-Wallis Test for testing groups with unequal sample sizes and unequal variances.
Almost any academic paper in the choice modeling literature will give you examples of the many ways folks conduct stat tests for their choice experiments.  I'm unaware of specific treatment about stat testing in conjoint analysis but for the chapter we included in our "Becoming an Expert in Conjoint Analysis" book.  That chapter is available separately here:  https://sawtoothsoftware.com/resources/technical-papers/statistical-testing