With full-profile presentation, respondents see concepts that are described on all the attributes. With full-profile presentation, there is a limit to how many attributes one should include in choice-based conjoint analysis. It is our opinion that concepts described by more than about eight attributes may confuse respondents and may cause them to respond superficially. But, it is difficult to establish a general limit, since the motivation and knowledge of respondents, the length of the attribute text, and the use of graphics affect how many attributes will work well.
If you must study more attributes than respondents can comfortably deal with in a CBC interview, you might consider some other method of conjoint analysis. Adaptive CBC may be a good alternative, since it is possible to drop entirely unimportant attributes from the product profiles. The Advanced Design Module for CBC can create partial-profile designs, where only a subset of the attributes is ever presented within a single choice task. Another conjoint technique, ACA (Adaptive Conjoint Analysis), can also handle larger numbers of attributes, though it has weaknesses especially in pricing research studies.
Number of Attribute Levels in CBC
The base CBC system lets you measure up to 15 levels for any one attribute (though the Advanced Design Module expands that number to 254 levels per attribute). Most projects will probably involve five or fewer levels per attribute, although attributes such as Brand (or SKUs reflecting brand x package combinations) may easily require many more levels for the purposes of a project. For typical CBC studies, it is usually better to have fewer levels on which attributes are described, along with approximate balance in the number of levels across attributes. With packaged-goods and beverage research, it may be reflective of real-world conditions to include dozens of levels to reflect brand or brand x package size combinations, but considerably fewer levels for variations in price.
We caution against using more than about five levels to define quantitative functions such as Price or Speed if utilities will be estimated for each discrete level (CBC's standard approach). CBC's market simulator permits interpolation between levels, so many additional points along a function may be estimated. Defining a quantitative attribute on too many levels spreads the data thin and increases the likelihood of getting reversals (out-of-order) utilities that are counterintuitive and problematic in simulations. Constrained estimation (monotonicity constraints) can help out in these situations. Another approach is to fit linear terms to quantitative attributes (also supported by our utility estimation programs). In that case, and if you believe the appropriate function is approximately linear, it may be justified to include more than five levels for quantitative attributes.
Number of Tasks in CBC
Because each respondent can receive a unique set of CBC tasks (questions) under randomized designs, as few as just one task may be asked per respondent while still allowing estimation of part-worths for the group (given a large enough sample size). Not surprisingly, most CBC questionnaires include multiple tasks, since it seems a waste not to collect more information from each respondent. With multiple observations per respondent, one can model heterogeneity in preferences, which leads to more accurate choice simulators.
In a 1996 meta-analysis of 21 CBC data sets, we found that multiple observations per respondent are quite valuable and that respondents could reliably answer up to at least 20 questions, and perhaps even more. However, we discovered that respondents process earlier tasks differently from later questions. Respondents paid more attention to brand in the first tasks and increased their focus on price in later questions. (See Johnson and Orme's article entitled "How Many Questions Should You Ask in Choice-Based Conjoint?" available for downloading from the Technical Papers section of our home page: http://www.sawtoothsoftware.com/support/technical-papers).
Over the last fifteen years, other researchers have examined the same issues and have suggested that respondents (especially internet and panel respondents) are probably less patient and conscientious with long CBC surveys involving many attributes than respondents of 25 years ago. They are perhaps quicker to resort to simplification heuristics to navigate complex CBC tasks and responses beyond about the tenth task don't seem to be revealing much more about each respondent's choice process. These authors have recommended using fewer than a dozen tasks, even if the attribute list is relatively long. They have recommended meeting the information requirements involved with long attribute lists by increasing the sample size.
With CAPI or paper-based CBC research, we recommend asking somewhere in the range of 8 to 15 choice tasks. With Web or mobile interviewing, fewer tasks might be appropriate if there is an opportunity for increased sample sizes (which often is the case). If estimating individual-level utilities using CBC/HB, we'd recommend at least six choice tasks to achieve good results based on simulated shares, but about 10 choice tasks or more for developing robust predictions at the individual level (again assuming a typical design). Before finalizing the number of concepts or tasks to be asked, we urge you to pretest the questionnaire with real respondents to make sure the questionnaire is not too long or overly complex.