Have an idea?

Visit Sawtooth Software Feedback to share your ideas on how we can improve our products.

Reconciling attribute importance with simulator output

I think the best way for me to ask this is with an example:

Let's say this is my design...
Ice Cream Flavor
-chocolate
-vanilla
-strawberry

Cookie Flavor
-sugar
-gingerbread
-snickerdoodle

Cookie Size
-big
-small
 
I’m interested in simulating choices between ice cream and cookies. I have a 3-group LCA analysis I want to work with.

Group 1:
Item type raw utilities: +2.06 ice cream, -2.06 cookie
Ice cream flavor raw utilities: -0.08 chocolate, -0.79 gingerbread, +0.79 strawberry
Cookie flavor raw utilities: +0.11 sugar, +0.11 gingerbread, -0.22 snickerdoodle
Cookie size raw utilities: +0.37 big, -0.37 small

When simulating each possible ice cream vs. cookie scenario, the probability of selecting ice cream in this group is between 0.97 - 0.99.

This makes sense to me. Item type is most important to this group. They would almost always choose ice cream, regardless of cookie characteristics.

Group 2:
Item type raw utilities: -6.05 ice cream, +6.05 cookie
Ice cream flavor raw utilities: +4.09 chocolate, +3.61 gingerbread, -7.70 strawberry
Cookie flavor raw utilities: +8.96 sugar, -4.79 gingerbread, -4.17 snickerdoodle
Cookie size raw utilities: +0.16 big, -0.16 small

When simulating each possible ice cream vs. cookie scenario, the probability of selecting ice cream in this group is 0.00 - 0.04

My question is how to reconcile this with the relative importances. Why is item type not also reflected as the most important attribute for this group? It seems like from the simulation projection, people in this group  nearly always select cookie over ice cream. Everything else only makes up for 4% of the variance.

Any help is thinking this through is appreciated! Thanks!
asked Feb 2 by anonymous

1 Answer

0 votes
You've got an alternative-specific design, where some attributes are not applicable to some alternatives.  Although our software calculates "attribute importances" for alternative-specific designs, they aren't as meaningful or applicable to interpret as for standard conjoint studies where all attributes are used to describe all product alternatives.

So, in general, don't try to interpret "attribute importances" for alternative-specific designs.
answered Feb 3 by Bryan Orme Platinum Sawtooth Software, Inc. (199,115 points)
Thanks for your response!

Something I forgot to mention is that the design is constrained to only show ice cream vs. cookie choices. So there's no cookie vs cookie, ice cream vs ice cream choices. So there are alternative specific constants, but every attributes is present during every choice trial.

Is it still true that importance shouldn't/can't be interpreted?
...