The Randomized First Choice (RFC) method combines many of the desirable elements of the First Choice and Share of Preference models. As the name implies, the method is based on the First Choice rule, and can be made to be essentially immune to IIA difficulties. As with the Share of Preference model, the overall scaling (flatness or steepness) of the shares of preference can be tuned.
RFC, suggested by Orme (1998) and later refined by Huber, Orme and Miller (1999), was shown to outperform all other Sawtooth Software simulation models (First Choice, Share of Preference, and Sawtooth Software no deprecated "Model 3" technique for correcting for product similarity) in predicting holdout choice shares for a data set they examined. The holdout choice sets for that 1998 study were designed specifically to include product concepts that differed greatly in terms of similarity within each set. In 2016, Sawtooth Software collected a CBC data set for the purposes of hosting a modeling competition that had 21 out-of-sample (completed by different respondents) holdout tasks featuring different degrees of product similarity. RFC predicted the holdouts better than all other standard Sawtooth Software simulation models.
Rather than use the utilities as point estimates of preference, RFC recognizes that there is some degree of error around these points. The RFC model adds unique random error (variability) to the utilities and computes shares of preference in the same manner as the First Choice method. Each respondent is sampled many times to stabilize the share estimates. The RFC model results in a correction for product similarity due to correlated sums of errors among products defined on many of the same attributes. To illustrate RFC and how correlated errors added to product utilities can adjust for product similarity, consider the following example:
Assume two products: A and B. Further assume that A and B are unique. Consider the following product utilities for a given respondent:
Avg. Product Utilities
A 10
B 30
If we conduct a first choice simulation, product B captures 100% of the share:
Avg. Product Utilities Share of Choice
A 10 0%
B 30 100%
However, let's assume that random forces can come to bear on the decision for this respondent. Perhaps he is in a hurry one day and doesn't take the time to make the decision that optimizes his utility. Or, perhaps product B is temporarily out-of-stock. Many random factors in the real world can keep our respondent from always choosing B.
We can simulate those random forces by adding random values to A and B. If we choose large enough independent random numbers so that it becomes possible for A to be sometimes chosen over B, and simulate this respondent's choice a great many times (choosing new random numbers for each choice iteration), we might observe a distribution of choices as follows:
Avg. Product Utilities Share of Choice
A 10 25.0%
B 30 75.0%
(Note: the simulation results in this section are for illustration, to provide an intuitive example of RFC modeling. For the purposes of this illustration, we assume shares of preference are proportional to product utilities.)
Next, assume that we add a new product to the mix (A'), identical in every way to A. We again add independent random variability to the product utilities so that it is possible for A and A' to be sometimes chosen over B, given repeated simulations of product choice for our given respondent. We might observe shares of preference for the three-product scenario as follows:
Avg. Product Utilities Share of Choice
A 10 20.0%
A' 10 20.0% (A + A' = 40.0%)
B 30 60.0%
Because in this illustration, unique (uncorrelated) random values are added to each product, A and A' have a much greater chance of being preferred to B than either one alone would have had. (When a low random error value is added to A, A' often compensates with a high random error value). As a simple analogy, you are more likely to win the lottery with two tickets than with one.
Given what we know about consumer behavior, it doesn't make sense that A alone captures 25.0% of the market, but that adding an identical product to the competitive scenario should increase the net share for A and A' from 25.0% to 40.0% (the classic Red Bus/Blue Bus problem). It doesn't seem right that the identical products A and A' should compete as strongly with one another as with B.
If, rather than adding independent random error to A and A' within each choice iteration, we add the same (correlated) error term to both A and A', but add a unique (uncorrelated) error term to B, the shares computed under the first choice rule would be as follows:
Avg. Product Utilities Share of Choice
A 10 12.5%
A' 10 12.5% (A + A' = 25.0%)
B 30 75.0%
(We have randomly broken the ties between A and A' when accumulating shares of choice). Since the same (correlated) random error value is added to both A and A' in each repeated simulation of purchase choice, A and A' have less opportunity of being chosen over B as compared to the previous case when each received a unique error component (i.e. one lottery ticket vs. two). The final utility (utility estimate plus error) for A and A' is always identical within each repeated first choice simulation, and the inclusion of an identical copy of A therefore has no impact on the simulation result. The correlated error terms added to the product utilities have resulted in a correction for product similarity.
Let's assume that each of the products in this example was described by five attributes. Consider two new products (C and C') that are not identical, but are very similar—defined in the same way on four out of five attributes. If we add random variability to the part-worths (at the attribute level), four-fifths of the accumulated error between C and C' is the same, and only one-fifth is unique. Those two products in an RFC simulation model would compete very strongly against one another relative to other less similar products included in the same simulation. When C received a particularly large positive error term added to its utility, chances are very good that C' would also have received a large positive error term (since four-fifths of the error is identical) and large overall utility.
RFC Model Defined
We can add random variability at both the attribute and product level to simulate any similarity correction between the IIA model and a model that splits shares for identical products:
Ui = Xi (ß + Eattribute) + Eproduct
where:
Ui = Utility of product i for an individual or homogenous segment at a moment in time
Xi = Row of design matrix associated with product i
ß = Vector of part-worths
Eattribute = Normally distributed error draws added to the part-worths (same for all products)
Eproduct = Gumbel distributed error added to product i (unique for each product)
Respondents are simulated to make repeated first choices for the simulation scenario with new random draws applying to each iteration. In RFC, the greater the variance of the normally distributed error terms added to the part-worths, the flatter the simulations become. The lower the variance of the error terms added to part-worths, the more steep the simulations become. Under every possible amount of attribute variability (with no product variability applied and all attributes set to receive correlated error), shares are split exactly for identical products, resulting in no "inflation" of net share. However, there may be many market scenarios in which some share inflation is justified for similar products. A second unique variability term (distributed as Gumbel) added to each product utility sum can tune the amount of share inflation, and also has an impact on the flatness or steepness of the overall share results. It can be shown that adding only product variability (distributed as Gumbel) within the RFC model is identical to the familiar logit model (Share of Preference Model), given enough draws (iterations). Therefore, any degree of scaling or pattern of correction for product similarity ranging between the First Choice rule and Share of Preference can be specified with an RFC model by tuning the relative contribution of the attribute and product variability. Stated another way, the First Choice and Share of Preference choice simulation rules are just special cases of RFC.
The exponent also can play a role in RFC, similar, but not identical to, product variability. Decreasing the exponent (multiplying the utility estimates by a value less than unity prior to adding the random error components) decreases the variance of the utility estimates relative to the variance of the random variation added within RFC simulations, in turn making simulated shares flatter. There is a subtle difference between increasing product variability and lowering the exponent, though both result in a flattening of shares. If only attribute variation (Eattribute) is being used in an RFC simulation, decreasing the exponent flattens the shares, but the overall model still does not reflect the IIA property. Adding product variability (Eproduct), however, flattens the shares and causes the RFC model to reflect at least some degree of IIA behavior. Though the exponent is not required to simulate different patterns of correction for product similarity and scaling, it is useful to retain the exponent adjustment from an operational point of view.
Number of Iterations
The RFC model is very computationally intensive. With the suggested minimum of 250,000 total RFC sampling iterations (across all respondents) for a conjoint data set, the results tend to be fairly precise. But, if you have dozens of products in the simulation scenario, some product shares can become quite small, and greater precision would be needed so that the remaining random component of RFC isn't unacceptably large compared to the signal component of the shares of preference. The market simulator makes some default adjustments to increase the number of sampling iterations as you increase the number of products in your simulation scenario:
Number of Products in Simulation Scenario |
Number of RFC Sampling Iterations across Respondents |
<10 |
250,000 |
10 to 24 |
750,000 |
25 to 49 |
1,500,000 |
50 to 99 |
2,500,000 |
100+ |
5,000,000 |
For example, if you had 500 respondents in your data set with 12 products in your market simulation, the RFC would conduct 750,000 separate iterations (shopping trips) across the 500 respondents, or 750,000 / 500 = 1500 iterations per respondent. The number of iterations used for each respondent is reported in the Simulation Settings tab in the simulator's output window. You can override the software's default settings to either increase or decrease the precision by specifying the number of sampling iterations to use per respondent. If you plan to use the RFC model with individual-level utilities to compute stable estimates of share at the individual level, you should sample each respondent at least 3,000 times, and preferably more. This can take a good deal of computing time.
Tuning the Variance of the Attribute and Product Error Draws
The greatest complexity of the RFC model from an operational point of view is that the magnitude of the attribute variability multiplier must be adjusted whenever the number of products or number of attributes on which products differ changes across simulations to maintain comparable scaling of shares of preference. Our implementation of the RFC model includes an auto-calibrating attribute variability multiplier (developed through Monte-Carlo simulations) so that this issue is transparent to the user. The multiplier used for each attribute is reported in the Simulation Settings tab in the simulator's output window.
Ideally, you will provide your own validation data, in the form of holdout concepts or actual market shares, to permit calibration of the attribute and product variability terms. This allows you to best fit the scaling of shares and degree of correction for product similarity appropriate for the specific market you are modeling. If you are using the auto-calibrating attribute variability multiplier, you can adjust the relative contribution of attribute and product variability by manipulating only the exponent and the product variability multiplier. For example, if you want to decrease the correction for product similarity by adding some product variability and adding less attribute-level variability to the utilities, you could increase the product variability multiplier (say, from 0.0 to 0.2). After doing so, the resulting shares of preference would be flatter than before the adjustment. (The more variation added to utilities, the flatter the resulting shares.) To "sharpen" the shares again, you would adjust the exponent upward. To adjust the amount of correction for similarity, you should have outside information about choices for holdout choice tasks, or actual market share, where the product concepts involved had significantly different degrees of product similarity within each set.
Attribute Error Draws for Interpolated Attributes
An additional technical issue arises for drawing appropriate ɛa error terms in the case of interpolation between levels of an attribute. If you interpolate between attribute levels, you must modify the Eattribute multiplier as follows:
a.Let a and b be adjacent levels involved in interpolation.
b.Let Fa be the fraction of the way from level a to b, and Fb the fraction of the way from b to a.
c.Additionally multiply the attribute-type error draws added to the utility of levels a and b by: 1 / Sqrt[(Fa2)+(Fb2)].
Recommendations for RFC
•The RFC model is appropriate for all types of conjoint simulations, based on either aggregate- or individual-level utilities. It provides the greatest benefits in the case of aggregate (logit and Latent Class) models, which are more susceptible to IIA difficulties than individual-level models. It provides typically modest benefits when you are conducting your choice simulations using individual-level utilities (such as from HB).
•Don't use RFC if only two products are included in a scenario; no correction for product similarity is needed and the results will be not very different from Share of Preference.
•Don't apply correction for product similarity (correlated Eattribute draws) for the price attribute, as this can result in strange kinks in derived demand curves (via sensitivity analysis). More information.
•Because RFC is much slower than Share of Preference, it can be difficult to use RFC for large product optimization search spaces. Do initial searches with a reduced number of RFC iterations per respondent by overriding the software's automatic settings. After initial searches, you can narrow down the search space to the most likely best outcomes and then increase the number of RFC iterations to achieve greater precision in the final solution.
•Recognize that RFC assumes that different levels within the same attribute are independent for each respondent. For example, if "Diet Coke" and "Coke Zero" are two different levels of the brand attribute, they will receive independent Eattribute draws in each iteration. Thus, no correction for similarity will be imposed within the unit of analysis (within the respondent, if using individual-level utilities). The only correction for similarity observed would come from the patterns of taste heterogeneity across respondents (e.g. the same respondents who like Diet Coke also tend to like Coke Zero).
How Does RFC Compare to Simulating on the HB Draws?
At Sawtooth Software, we are practitioners who tend to favor speed, simplicity, and practicality. Since the late 1990s the default way we have used utilities estimated via HB in Sawtooth Software's simulators has been to use the point estimates (one vector of utilities) per each respondent. These point estimates are averages of the typically 1000s of draws of the respondent utilities after convergence is assumed. We came to the determination to use point estimates rather than draws in our choice simulators based on favorable comparisons we made in the late 1990s, and also influenced by the slower speed of computers at the time.
In 2016, we compared the predictive validity of RFC and HB draws using a very strong CBC data set that included 21 holdout choice scenarios designed with differing degrees of inter-product similarity, completed by out-of-sample respondents (different respondents from those used to estimate the conjoint utilities). Simulating on the HB draws slightly edged out RFC 0.909 to 0.907 (where these are correlations of population-level predicted shares of preference to actual out-of-sample shares of choice for the holdout scenarios). The correlation between RFC and HB draws in terms of population-level predictions was 0.997.
Now after years of hindsight, we recognize that not only are computers much faster now, but it is more formally correct and complete to conduct simulations on the draws rather than the point estimates of preference. For certain data conditions, simulating on the draws may provide modestly superior results relative to RFC. HB draws for each respondent tend to be distributed normally, but with variances and covariances resembling the pattern of variances and covariances estimated in the upper-level population covariance matrix. In contrast, RFC with its IID (Independent and Identically Distributed) error draws is a simplification of the empirically estimated uncertainty and covariances as estimated via HB. Despite the simplification, RFC generally works quite well in practice and our implementation of RFC within the Choice Simulator operates much faster than utilizing a similar number of HB draws per respondent.
If we were tasked with presenting simulation results to an academically-minded audience, and if we had the luxury of greater time to conduct the analysis, we would probably take the approach of simulating on the HB draws rather than using RFC. The results would be extremely similar, but more proper and defensible.
For more information about the RFC model, please refer to Huber, Orme and Miller's paper entitled, "Dealing with Product Similarity in Choice Simulations," available for downloading from our home page: http://www.sawtoothsoftware.com.