MaxDiff RLH with 9 sets of 6 items each and each item shown 3 times

Question

We ran a MaxDiff with 18 items.  We used 9 sets of 6 items each and with each item seen 3 times.  N=150.
The RLH is running from .17 to .58.  What should the minimum RLH be to for good respondents?
Thank you!

Answer 1

Here's the initial paper on the topic which provides your answer:  https://www.linkedin.com/pulse/identifying-consistency-cutoffs-identify-bad-respondents-orme/?trackingId=g9ylG8GeasHXn79cZcC5JQ%3D%3D

And here's a recent update:  https://sawtoothsoftware.com/resources/technical-papers/diagnostics-for-random-respondents-in-choice-experiments

Comments

Thank you.  Based on the numbers I shared, is an RLH of .17 too low?

---

Probably, yes.  It's essentially the fit you'd expect from random responders (6 items/set you'd guess random data correctly 1/6 = 16.667% of the time).