Have an idea?

Visit Sawtooth Software Feedback to share your ideas on how we can improve our products.

How to compare attribute importance at different time points.

Hey,
I'm going to use a hypothetical example to explain my question.

I want to know whether people prefer riding a bike to work or taking the bus. I want to survey the same group of people four times during a year: August, November, February, and May.

Bike: Alternative constant ($0)

Bus:
Crowdedness: High, medium, low
Length of ride: Shorter than biking, Longer than biking

I want to know how time of year affects choices. I predict that people will be more likely to bike in August and May (due to daylight and weather). In November and February though,  I think bus crowdedness will have higher importance than it does in August and May.

So a few questions around this kind of scenario...
A) How can I compare the importance of bus crowdedness in August to bus the importance of bus crowdedness in February? Could I test the the two importance proportions (McNemar's or Cochran's Q)? Or would it be better to compare the utility estimate of a single level at both time points via t-test?

B) I'm lost as to a minimum acceptable sample size. I've read the sections you have on this topic - I know it depends on the design complexity, etc. But I'm interested in this time of year comparison most of all, which is a slightly different question than usually addressed with CBC. For feasibility/budget purposes, I'm interested in getting that number as low as possible.
asked 6 days ago by Cassidy

1 Answer

0 votes
Cassidy,

I would use the importances, not the utilities (which latter are more affected by differences in the logit scale parameter).  Importances are metric variables, so a t-test would be appropriate for 2 groups/time periods, and ANOVA would be appropriate for 2+.

As for sample size, I don't know anyone who DOESN'T want to make it as low as possible.  But you have more power to detect differences with more respondents, as with any other metric variable in any other stat test.  If this is the single more important comparison you're making, I would in your shoes want at least n=300 in each group, so as to have plenty of power for the comparison.
answered 6 days ago by Keith Chrzan Platinum Sawtooth Software, Inc. (95,175 points)
Thanks for the quick response.

Does the alternative constant get an importance as well?
bike importance + bus crowdedness importance +bus length of time importance = 1?
Bike in your model will get a utility, but not an importance.
So I wouldn't be able to comment on whether general transportation type or bus crowdedness was more important?

Could I potentially conclude something like bikes have a greater utility in august than even the most appealing bus?
It depends a little on how you code the experiment, but I think you could compute importance of mode as follows:  If you think of it as an alternative-specific design, you would have a utility for bus and a utility for bike; if you code this where bike is the "none" or "other" alternative, then the utility for bus is fixed at 0 and that for bike is calculated.  Either way, you can compute an importance for mode, if you like.
I think this is my last question. I appreciate your help so much.

Would I be able to comment on the importance of mode vs the importance of bus crowdedness? If one goes up, does the other go down?
Not necessarily, because you also have the length of ride, right?
Right, but do those three importances (mode, crowdedness and length of ride) add up to 1?
Yes, if you normalize them properly, they will.
...