Have an idea?

Visit Sawtooth Software Feedback to share your ideas on how we can improve our products.

RLH benchmark

With the past projects I have sometimes run into data that sometimes show an average RLH in the 300-400 levels.  I was wondering if there is a hard cut off that you would decide that the HB result is not reliable, or that the conjoint has some basic design issues?
related to an answer for: Improving RLH
asked Jun 13, 2014 by HongHu Bronze (835 points)

1 Answer

0 votes
Best answer
RLH (root likelihood) depends strongly on the number of concepts shown in each task.  If just 2 concepts are shown, then the chance RLH rate is 0.5.  With 10 concepts per task, the chance RLH rate is 0.1.  So, you cannot compare RLH between studies unless you somehow adjust for number of concepts shown per task.  Pct. Crt. does that.  It can be treated like a pseudo-Rsquared.

In a recent training, I reported Pct.Cert results for HB estimation for 25 commercial and methodological CBC studies (standard first-choice CBCs...not DRNone, chip allocation, or B-W responses).  I had a mean Pct. Cert across those studies of 0.71, with a min of 0.60 and a max of 0.83.  

So, you want to be in that range.
answered Jun 13, 2014 by Bryan Orme Platinum Sawtooth Software, Inc. (175,315 points)
selected Jun 13, 2014 by HongHu
That makes great sense.  I believe that chip allocation may allow a lower pct cert, correct?  What about ACBC and MaxDiff?  Should we expect a higher or lower minimum pct cert?
RLH in MaxDiff