Randomizing Question/Pages/Blocks
There are times where the researcher wants to randomize a series of pages, questions on a page, or blocks of questions. This is most often done to reduce response bias due to question order. Question order bias occurs when respondents answer differently to later questions because of the influence of previously answered questions. If questions are displayed in random order, question order bias still exists at the individual level. But, because individuals each receive a different question ordering, the bias is nearly eliminated when viewing aggregate results.
You can randomize the order that questions are asked within the same page. You can also randomize the order of pages. You cannot randomize the order of questions that span from one page to another, when there are multiple questions per page.
Randomizing Questions on a Page
To randomize the order that questions are asked on a page:
1) Click Compose | Write Questionnaire and click the Randomize... | Questions on a Page button. From the Randomize Questions on a Page dialog, click the Add... button.
2) Select the beginning and ending question. All questions from the beginning to the ending question (including the beginning and ending questions) are displayed in random order when the questionnaire runs on a Web server.
Randomizing the Order of Pages
1) Click Compose | Write Questionnaire and click the Randomize... | Pages button. From the Randomize Pages dialog, click the Add... button.
2) Select any question from the first and last pages in the randomized sequence. All pages including and between the anchor questions are displayed in random order when the questionnaire runs on a Web server.
Randomizing Blocks of Questions
Example: imagine a questionnaire with three blocks of questions Q1-Q3 (block 1), Q4-Q10 (block 2), and Q11-Q20 (block 3). We may want to randomize the order that those three blocks of questions occur. For example, one possible ordering is Q4-Q10, Q1-Q3, Q11-Q20.
1) Click Compose | Write Questionnaire and click the Randomize... | Blocks button. From the Randomize Blocks dialog, click the Add Set button.
2) Add each block of questions by clicking Add Block... and then choosing a beginning and ending question for each block.
3) Select to either randomize the order that the blocks are asked, or to ask the blocks in the order as indicated in a constructed list. For example, if the blocks were to be asked in the order as represented within a constructed list and that list contained elements ordered 3, 2, 1 then the blocks would be asked in order: block 3, block 2, block1.
Notes: Each block must be separated by at least one page break. Questions can be additionally randomized within blocks.
Showing a Random Subset of Questions/Blocks
Example: imagine a questionnaire with 10 questions (or blocks of questions). You wish to randomize their presentation and have respondents complete a randomly selected 5 of the 10 questions.
1) Within Compose | Lists..., create a list called list10 with 10 members (you may just use integers 1 through 10 as the list member text). Add a constructed list called randlist5, that uses parent list list10. Use the following constructed list logic to randomize the list10 members and trim the list to just 5 members:
ADD (list10)
RANDOMIZE ()
SETLISTLENGTH (5)
2) Click Compose | Write Questionnaire and click the Randomize... | Blocks button. From the Randomize Blocks dialog, click the Add Set button.
3) Add each question, or block of questions, by clicking Add Block... and then choosing a beginning and ending question for each block. (If you are selecting random subset of single questions, the beginning and ending question for each block will refer to the same one question.)
4) Select to radio button (Use Constructed List) to randomize the order that the blocks are asked according to the constructed list, randlist5. When the questionnaire runs, if the constructed list contained elements 4, 2, 8, 7, and 1 the question blocks would be asked in order: block 4, block 2, block 8, block 7, and block 1.
Notes: Each block must be separated by at least one page break.