Have an idea?

Visit Sawtooth Software Feedback to share your ideas on how we can improve our products.

how to calculate the Hit Rate


I was wondering after recreating the hold out tasks in the simulator and running the specified model to predict the answers, How can I compare it to the actual answers and calculate the hot rate?

related to an answer for: Hit Rate & Segments Report
asked May 26, 2020 by AMYN Bronze (3,000 points)

1 Answer

+1 vote
Best answer
From the simulator, export respondent level choice predictions and assume the alternative with the highest share is the one chosen.  

Now, let's say you had 3 holdout tasks.  For each one that the prediction matches a respondent's choice, that's a hit.  So if for a given respondent Jones your model correctly predicts one of the three choices, your hit rate for that respondent is 33.33%.  If it was 3 hits or 0 then it is 100% or 0% and so on.  Now average that across respondents.  Voila, a hit rate.
answered May 26, 2020 by Keith Chrzan Platinum Sawtooth Software, Inc. (102,700 points)
selected May 27, 2020 by AMYN