Although CVA's question format has advantages in studies with small numbers of attributes, its task can be challenging for the respondent. To be sure that your CVA questionnaire is appropriate for the study, you must:
•keep the number of attributes small,
•pretest the questionnaire, and
•conclude that the resulting utilities are reasonable,
before you proceed.
Any conjoint questionnaire can be difficult for the respondent, but CVA questionnaires can be particularly so. Unlike our ACA System for Adaptive Conjoint Analysis, CVA does not use any strategy for simplifying the respondent's task. Every question involves consideration of all the conjoint attributes. CVA can create either single concept (card sort) or pairwise comparison (two concepts at a time). With pairwise questions, the amount of information the respondent must consider is twice as great as with single concept methods.
Unless the number of attributes is quite small, (no more than about six to eight) there is risk that respondents may become overloaded, and that the quality of the data collected may suffer. For this reason, we recommend other types of questionnaires if many attributes must be studied.
In summary, although the CVA question format can be very effective when the number of attributes is not too large, we are concerned that inappropriate use of CVA may produce questionnaires that are too difficult for respondents. As with any conjoint method, but particularly so in this case, it is essential to pretest the questionnaire:
•answer the questionnaire yourself, and analyze your own data to make sure the part-worth utilities mirror your own values.
•have others answer the questionnaire to report to you about whether it's too difficult.
•have a sample of relevant respondents answer the questionnaire and analyze their data. Be particularly alert for "nonsense" results with respect to price. If many respondents have higher utilities for higher prices, then the data are suspect.
If you do keep the number of attributes small and if you do pretest the questionnaire, concluding that the resulting utilities are reasonable, then you can proceed with confidence.