Have an idea?

Visit Sawtooth Software Feedback to share your ideas on how we can improve our products.

Order of removing bad respondents


I want to remove bad respondents by looking at the RLH, sys_SumPageTimes and straightliners. However, I was wondering about the order of removing the respondents.

Should you first remove respondents based on RLH, then remove the straightliners, and then after that calculate the median survey time based on that new dataset where the respondents that have RLH values lower than the 95th percentile and the straightliners are already removed after which you remove speeders?

Or is it better to see which respondents you need to remove based on the RLH, see which respondents you need to remove based on the time (where you calculate the median time across all complete ones, so before removing the ones based on RLH), see which respondents are straightliners and then after that compare which respondents you would need to remove in the 3 cases and remove those? (so maybe there is a respondent who needs to be removed based on the RLH as well as time)?
asked Apr 21 by 123

1 Answer

0 votes
You can remove straightliners even during the survey, right?  In any case, I think those are the first I would delete.  

Usually I don't remove respondents simply for speeding or simply for speeding, but I use a "two strikes" rule and remove those with low RLH and for evidence of inattentiveness (like speeding).
answered Apr 21 by Keith Chrzan Platinum Sawtooth Software, Inc. (102,700 points)
Ah okay, great! Thank you for your answer :)

I have one more question: if you calculate the median time spent, would you then include the outliers (in terms of sum page time) in calculating your median or not?