Calculating Hit Rate Using SSI Simulator

Question

Hi,

I'm interested in predicting in-sample hit rates and I understand that this can be achieved by using SMRT. However I am wondering if there is a way to calculate hit rates using the SSI simulation module rather than export files to SMRT. If there is how would this be achieved?

Thanks

Jasha

Answer 1

I think you're probably referring to our new choice simulator, built into Lighthouse Studio, or also available as a standalone product called "Sawtooth Software Choice Simulator."

Add as many scenarios to the simulator as you have holdout tasks.  Select the "First Choice" rule for each scenario if you want the 1/0 hit rate metric.  Make sure you are using an individual-level utility run, such as produced by HB.  On  each "My Scenario Settings" tab for each of your scenarios, make sure to click the "Individual Results" icon in the Reporting group on the ribbon.

When you run each simulation scenario, one of the tabs that is shown in the reporting area is called "Individual Results."  This reports for each respondent which of the products in the simulation scenario the respondent would be most likely to choose.  You'll need to take that info into another program like Excel, SPSS, R, etc. and compare the predicted choice to the actual choice for the holdout from the questionnaire data that you have exported from Lighthouse Studio (or wherever you collected the holdout data).  If the predicted choice matches the actual choice for that holdout, then you score it a hit (1); if not, then you score it a miss (0).  Averaging the hits across respondents for each scenario provides the % hit rate.

Comments

Hey Bryan,
I have done as you describe above to get the predicted choice with the SSI simulator... Now I am trying to compare the simulated choice with the actual choice. But I am not sure where to get my actual choice.  Do I find it in the HB Excel output or somewhere else?

---

You can find answers to the fixed choice tasks through the regular data export options in lighthouse studio, for exporting all the raw answers to the questions in the survey to a .CSV file.  File + Data Management + Export Data (tab)