The Format tab on the MaxDiff Exercise dialog provides many options for formatting your MaxDiff or Method of Paired Comparison (MPC) questions. We suggest you experiment with the settings and click the Preview button to view the effects.
There has not been much research conducted on the best layout for MaxDiff questions. We provide different layouts here for your experimentation and use. If these options do not give you the exact look you need for your question, you can customize your own MaxDiff layout using Free Format and your own scripted HTML.
One significant methodological question is whether to ask for both "best" and "worst" (when displaying three items or more per set) or whether to omit the "worst" question.
Should We Ask for "Worsts"?
MaxDiff allows researchers to ask for "best" and "worst" choices within subsets of items (set size >=3), or to ask only for "bests." Collecting both bests and worsts contributes more information. However, it has been shown that the scores resulting from best choices may differ (statistically significant differences) from those developed only using worst choices. However, the results tend to be quite similar between bests and worsts. There is some debate among leading academics regarding the statistical properties of "worsts" and whether including both bests and worsts is appropriate. It would seem to depend on the purpose and context of the research. Some types of research need to focus on how to avoid "worsts" for people, whereas other types of research would focus on how to give people "bests".
In general, asking only for "bests" seems more theoretically sound for most research applications, but asking for "worsts" seems to offer practical value for stabilizing scores with low additional respondent effort. We hope that offering flexibility in this software will lead to more experimentation in this area.
Anchored Scaling: Direct and Indirect Methods
Respondents indicate which items are relatively better (or worse) than others within standard MaxDiff questionnaires. Thus, the scores are estimated on a relative scale, without any indication that the items are good or bad, important or unimportant, in an absolute sense. So, a respondent who dislikes all the items could have final HB scores that look essentially identical to another respondent who likes all the items a lot. Including a broad range of items in your study (from generally low to high importance/preference) helps reduce these problems. Inserting an item that has concrete monetary meaning or conveys a status quo can also be used to anchor the scores. MaxDiff's limitations related to the relative scaling of items have been debated for years now and two additional approaches to anchored scaling have emerged: the Indirect and Direct anchoring methods. Read more about Anchored Scaling.