The following survey was conducted for my university thesis.

I did a CBC consisting of 10 random tasks and 2 fixed tasks.

There were 3 stimuli with no none option per task. In addition, there were 5 characteristics, each with a positive and negative value.

I did not create the Hold-out Tasks according to the recommended 50/30/20 rule, but more or less arbitrarily. Very stupid, I know. I didn't know better...

In the first fixed task, I presented in each stimuli three characteristics in positive expression and two in negative expression.

When I now threw out the utility values via Sawtooth and then reconstructed my hold-out tasks in Excel to determine the highest utility values and based on this my hit rate, I noticed that often 2 of the 3 possible alternatives have the same highest part worth.

Therefore, I cannot determine whether it was a hit because, for example, although the respondent chose stimuli 1, both stimuli 1 and 2 have the (same) highest part worth.

What should I do now or is there an error somewhere in the calculation? Can this still be saved?

I would be incredibly grateful for help!

thanks for your ideas!

I have a question: Did I even do it the right way?

I went to Sawtooth Lightstudio -> Analysis -> did a Hierarchical Bayes (HB) -> took the Utilities (Zero-Centered Differences) to Excel -> Then built my Hold-Out Tasks together in Excel and compared the highest value(s) with the real answers of the participants.

Was this the right way to do calculate the Hit-Rate?

Thank you very much!