how can I tune the exponent in the market simulator (using the "share of preference model") if I have not used any holdout tasks in my CBC?

In this case, is it better to keep the default (exponent: 1) and not adjust the exponent?

Thank you!

Visit Sawtooth Software Feedback to share your ideas on how we can improve our products.

how can I tune the exponent in the market simulator (using the "share of preference model") if I have not used any holdout tasks in my CBC?

In this case, is it better to keep the default (exponent: 1) and not adjust the exponent?

Thank you!

+1 vote

Best answer

In fact, your SHOULD be ready to tune the exponent in the absence of holdout tasks! Tuning the exponent principally would be helpful to adjust for differences in response error between survey respondents and real world buyers making choices. If you have evidence regarding real world choices (such as actual market shares), then this could be used to adjust the exponent.

The idea of tuning the exponent using a few holdout choice tasks that were asked of the same respondents in the same survey as the utility estimation tasks is generally not helpful. The response errorwould be expected to be the same between the tasks used for estimation and the holdout tasks that were mixed among them (assuming the layout and complexity of the choice tasks were the same between holdouts and utility estimation tasks). Thus, the scale factor (exponent) obtained naturally (exponent=1) through utility estimation on the many tasks used to estimate the utilities should be more accurate than a new scale factor obtained by tuning the utilities to fit the fewer holdout tasks.

The idea of tuning the exponent using a few holdout choice tasks that were asked of the same respondents in the same survey as the utility estimation tasks is generally not helpful. The response errorwould be expected to be the same between the tasks used for estimation and the holdout tasks that were mixed among them (assuming the layout and complexity of the choice tasks were the same between holdouts and utility estimation tasks). Thus, the scale factor (exponent) obtained naturally (exponent=1) through utility estimation on the many tasks used to estimate the utilities should be more accurate than a new scale factor obtained by tuning the utilities to fit the fewer holdout tasks.

...

Unfortunately I have no evidence regarding real world choices, because the product is very special.

Is an adjustment without "real world data" not possible? Is there a possibility to adjust the exponent based on the part worth utilities or should I use the default?