Have an idea?

Visit Sawtooth Software Feedback to share your ideas on how we can improve our products.

p-value from t-ratio


I am doing a CBC analysis for my thesis. Now I have run a logit analysis, and I have learned that an attribute is significant if it has at least one level with an absolute t-value > 1.96. However, my supervisor wants me to report the p-values instead of the t-values. For example, I have to write 'X has a significant influence on Y (p < ...)' instead of 'X has a significant influence on Y (t-ratio > 1.96 for all levels)'. But I was wondering, how could I compute an overall p-value per attribute based on the t-ratios?
asked May 28 by 123

1 Answer

0 votes
If your supervisor needs to have an overall p-value per attribute rather than per level...and if you're confined to using aggregate logit analysis, then I can think of the following approach:

1.  Run aggregate logit with all attributes included in the model.  Write down the log-likelihood of the model.
2.  Run the model again, after omitting the first attribute.  (You can omit an attribute by setting its attribute coding to "excluded").  Write down the log-likelihood of the model.
3.  Take the difference in log-likelihoods between steps 1 and 2 and multiply that difference by 2.  That result is distributed as chi-squared.  The stats test involves that chi-squared value as a critical value, with degrees of freedom equal to the number of levels in the excluded attribute minus one.  The chi-squared test of course produces a p value.
4.  Repeat steps 2-3 omitting each attribute one at a time from the full model.  (So, if you have 5 total attributes in your study, you'll be doing this 5 separate times, where one attribute is omitted each time from the full model.)

The main flaw to this approach is if you have an attribute like brand or color for which respondents are in perfect disagreement about.  For example, imagine an attribute like color with just two levels: Red and Blue.  Imagine half the respondents prefer Red and the other half Blue (to an equal degree).  In the aggregate, these two levels will cancel out in utility and the fit improvement in an aggregate logit model will be zero from such an attribute.
answered May 28 by Bryan Orme Platinum Sawtooth Software, Inc. (189,140 points)
I should clarify that I'm assuming that you're using the default part-worth (effects-coded) model, such that an attribute with (say) 5 levels involves 4 degrees of freedom.  If you are using linear coding (one parameter per attribute), then there is a difference of just one degree of freedom involved when performing the omit-one-attribute statistical test I described.
Thank you for your great answer Bryan, very clear! Now I know how to do it.
Yes, I am using effects coding, thanks for your clarification!