# Significance testing

Hello,

I’ve done a CBC experiment and want to conduct now some tests and see if results are significant.
I read a lot about this topic in the forum and the book chapter “Statistical testing” and wanted to make sure that I understood it right (I decided to go for frequentist tests):

1. Significance of attribute levels / significance of attribute importances

Option 1 – Calculation of t-ratio/Hypothesis testing (estimate unequal 0)
Test: Calculate t-ratio (estimate divided by SE) – if absolute value >1.96, p < .05
Data: Average utilities/Average importances

Option 2 – Confidence Intervals
Test: Look if confidence intervals do not contain 0
Data: Average utilities/Average importances

Rather than spend time testing whether levels/attributes have a significant effect in conjoint analysis (e.g., effect coding can be misleading, attributes mostly significant from 0), you should concentrate on differences

2.1 Significant difference between attribute levels (attribute with 2 levels)

Test: Dependent t-test  (if assumption of normality of differences violated: Wilcoxon Signed-Rank Tests)

Data: Individual level point estimates (zero-centered diffs)

2.2 Significant difference between attribute levels (attribute with 3 levels)

Test: One-way repeated measures ANOVA with dependent t-tests as post-hoc tests - Bonferroni correction to be applied (if assumptions of normality and sphericity violated: Friedman’s ANOVA with Wilcoxon Signed-Rank Tests as post-hoc tests)

Data: Individual level point estimates (zero-centered diffs)

3. Significant different between attribute importances (more than 2 attributes)

Test: One-way repeated measures ANOVA with dependent t-tests as post-hoc tests - Bonferroni correction to be applied (if assumptions of normality and sphericity violated: Friedman’s ANOVA with Wilcoxon Signed-Rank Tests as post-hoc tests)

Data: Individual level importance scores