Have an idea?

Visit Sawtooth Software Feedback to share your ideas on how we can improve our products.

RLH and Percent certainty

Hello,
Do you know how to explain RLH and Pct.Cert ?  If RLH is 0.6 the other one is o.5.
asked Dec 18, 2016 by anonymous

1 Answer

+1 vote
Using the logit equation you can predict the probability of choice for each of the alternatives (concepts) in a choice task for each respondent.  RLH (Root Likelihood) is the geometric mean of the predicted likelihoods for the concepts the respondent actually chose.  

So, if there were just three choice tasks and according to the logit equation the likelihood of the respondent picking the concept she actually did for each of the three tasks happened to turn out as 0.5, 0.7, and 0.8, then the RLH would be (0.5x0.7x0.8)^(1/3) = 0.654.

Pct. Cert. is a related statistic, but is computed differently.  It has a more intuitive meaning (it's also known as pseudo R-squared) that essentially says "what percent of the way between the null prediction and the best possible prediction did we get with this model?"

But, rather than use likelihoods, we use log-likelihoods (LL) to compute this statistic.  I'll illustrate.  Following from the previous example, imagine that each choice task had four alternatives.  The null likelihood for each task (based on a chance prediction) would be 0.25.  The null log-likelihood for each task would be ln(0.25).  The total null log-likelihood across three tasks would be 3(ln(0.25))=-4.159.  So, that's the floor (the worst fit in terms of LL we'd expect from a model that didn't fit any better than the chance level).

We previously said the logit equation gave us the likelihood that the respondent picked each of her three tasks as 0.5, 0.7, and 0.8.  The log-likelihood would then be ln(0.5)+ln(0.7)+ln(0.8) = -1.273 for our model.

The ceiling (the best possible prediction) for these three tasks would have a log-likelihood of 3(ln(1.0))= 0.000.  That occurs if our model could predict each of the three concepts the respondent actually chose (according to the logit rule) at a rate of 100%.  

Thus,

LL Floor = -4.159
LL our model = -1.273
LL perfect prediction = 0.000

Therefore, our model has a pseudo R-squared of (-1.273- -4.159)/(0.000- -4.159) = 0.6939.  Meaning, our model predicts at a rate 69% of the way between the worst expected and the best possible prediction.
answered Dec 19, 2016 by Bryan Orme Platinum Sawtooth Software, Inc. (198,315 points)
...