Most popular tags

cbc lighthouse-9 acbc ssi-web ssi-web-8 maxdiff cbc-hb javascript grid-question constructed-list unverified-perl ssi-web-7 free-format skip-logic alternative-specific-design willingness-to-pay select-question choice-simulator cbc-latent-class quota-control acbc-hb mbc conditional-pricing smrt pass-in-fields

Have an idea?

Visit Sawtooth Software Feedback to share your ideas on how we can improve our products.

Low Percent Certainty and RLH

I have conducted a CBC study and when I went and use HB to estimate the utilities, I got a low number for the percent certainty and RLH (Pct. Cert = 0.342 & RLH = 0.402) with a total of 152 respondents.

Is there any way that I can improve these numbers?

How can I identify respondents that lower those numbers and eliminate them?

Thank you.
asked Jul 22, 2021 by Thara

1 Answer

0 votes
Thara,

To answer your second question first, you can check out this white paper on our website:  https://sawtoothsoftware.com/resources/technical-papers/consistency-cutoffs-to-identify-bad-respondents-in-cbc-acbc-and-maxdiff

However, I'm not sure you need to go that route just yet, because I'm not sure your RLH is all that low.  How many alternatives did each respondent have to choose from in each choice set?  That number affects RLH.  If you had 5 or more alternatives per choice set, then your RLH looks quite good to me and you're trying to solve a small problem.  But if it was fewer than 5 then yes, you might want to identify and discard some random respondents.
answered Jul 22, 2021 by Keith Chrzan Platinum Sawtooth Software, Inc. (111,275 points)
There are 4 alternatives (including the "none" option) in each choice task. Does that mean my RLH number has to be greater than 1/4 = 0.25?

Do I have to include the "none" option?
commented Jul 22, 2021 by Thara
A random RLH would be around 0.25, yes, so clearly your data has a lot of non-random respondents.  I usually like my sample RLH to be at or near double the chance rate, so at or near 0.50 in your case.  You're close to that, but you might want to identify and consider removing random respondents.
commented Jul 22, 2021 by Keith Chrzan Platinum Sawtooth Software, Inc. (111,275 points)
Alright! Thank you so much for your reply, Keith. I removed some random respondents and the numbers went up by 0.2.
commented Jul 23, 2021 by Thara
Oh, very good.  Sounds like it worked well for you.
commented Jul 23, 2021 by Keith Chrzan Platinum Sawtooth Software, Inc. (111,275 points)
Powered by Question2Answer
...