Experimental Design Creation Settings
Random Tasks
The Random Choice Tasks are the questions/scenarios generated by CBC's design algorithms for the purposes of estimating part-worth utilities. The attribute levels making up the product concepts are formulated to have good statistical properties (such as level balance and independence of attributes). It is typical to include about 8 to 15 of these types of choice tasks in your study.
Fixed Tasks
If you want to specify specific choice scenarios for CBC to ask of every respondent, these are called Fixed Choice Tasks. If you add these to your questionnaire, you must specify the attribute level combinations that define the tasks (by clicking the Fixed Task Designs... button). Fixed Tasks are often used to include scenarios of particular interest to the client and for collecting additional choice tasks that may be used to assess internal validity. If you are not inclined to include any fixed tasks in your study, you may set the number of fixed tasks to 0.
Concepts per Task
Here you specify how many product concepts (alternatives) to include in each choice task. Although it is tempting to want to make it easier on respondents by displaying fewer concepts per task, we encourage you to ask respondents to consider many product concepts per task. For example, if each of your attributes had 3 levels, it would be a good idea to show 4 or 5 product concepts on the screen, so that some levels repeat within the choice task. This not only means each choice task provides more information, but including level overlap encourages respondents to provide more thoughtful responses and leads to stronger estimation of potential interaction effects.
None Option
You may include an option for respondents to specify that they would choose None of the products. This option is sometimes referred to as the "constant alternative" and may also be used to specify a status quo state, such as "I would continue with my current provider rather than switch to one of these plans."
With the "Traditional" None Option you can specify whether the None Option be displayed as a "Concept below the task", as the "Last concept in the task", or in the CBC question's "Header or Footer". If you select "Header or Footer" you will need to place the Sawtooth Script "[%CBCNone()%]" in the header or footer of the question.
A "Dual Response None" option is also available, wherein respondents first must select a product concept and are next asked whether they would actually purchase that product or not.
For more information see: None Option/Dual-Response None
Random Task Generation Method
This control lets you select which design method CBC should employ in determining the combinations of attribute levels to be shown in the Random Choice Tasks for your survey. We encourage you to read the section on CBC Questionnaires and Design Strategy to learn more about CBC's design strategies. For standard CBC studies, we generally suggest using either the Balanced Overlap or Complete Enumeration design strategies. (Note: these methods are computationally intensive, and for very large numbers of attributes and levels may require a long time to generate a design file.)
Design Type
Those with the base CBC license alone are limited to Traditional Full-Profile CBC Designs. Users with the Advanced Design Module may also
choose Alternative-Specific Designs and Partial-Profile Designs. These approaches are described later in this documentation.
Questionnaire Versions
CBC automatically creates a design file that is a) uploaded to the server if using web-based data collection, b) is copied to other laptops/devices if using CAPI-based data collection, or c) is used to generate a questionnaire file if using paper-based data collection. During the process, you are asked to specify how many versions of the questionnaire should be written to that design file. For computer-based interviewing, we generally recommend that you include at least 100 versions, as there are benefits to fielding many versions of the questionnaire. Optimally, each respondent would receive his/her own version of the questionnaire. However, there are diminishing returns and including more than about a dozen versions often is of very little to no additional practical benefit. For that reason, we limit the number of versions that can be placed in the design file to 999. If you have more than 999 respondents, this means that some respondents will by necessity receive the same designs. For all practical purposes, once there are so many unique versions of the questionnaire in the total design pool, this poses no harm.
Design Seed
This is the seed value used in the random number generator for the design algorithm. By using a different starting seed, a different design will result. With very many versions of the questionnaire, the overall design efficiency should be nearly identical no matter the random seed. With only a few versions of the questionnaire, using a different starting seed can have a meaningful effect on the overall efficiency.
Attribute Randomization
Attribute Randomization specifies whether to present the attributes in random presentation order within a concept. If randomization is enabled, the attribute list is randomized once per respondent, and all tasks within a respondent's interview will display the attributes in that given order. This can be useful to control order effects across respondents.
If Randomize All is selected, then all attributes are randomized. If Randomize a Subset is selected, then the Settings... button allows you to specify the range of attributes to randomize (the attributes outside the range are not randomized).
First Randomized Attribute
The first attribute in the range of attributes to be shown in random presentation order. Specify a "1" if you wish all attributes to be randomized. If, for example, you want the first and second attributes always to appear as the first two attributes in a product concept, specify a "2."
Last Randomized Attribute
The last attribute in the range of attributes to be shown in random presentation order. Specify the last attribute number if you wish all attributes to be randomized. If, for example, you had a total of five attributes in your study and you wanted the fourth and fifth attributes always to appear as the last two attributes in a product concept, specify a "3."
Concept Sorting
Concept sorting refers to the order in which the product concepts (profiles) are displayed in a task. For example, consider a choice task with three product alternatives. The default behavior in CBC software is to randomize the order in which these product concepts are displayed. Thus, a given brand may appear in the first concept in one task, but in the last concept position in the next task.
However, researchers do not always want the levels appearing in random concept positions within the task. There are two options for researchers to control concept ordering:
Randomized Level Order. With this option, the researcher selects (using the Settings... button) a specific attribute that is used to control the concept presentation order. For example, the researcher may choose the "brand" attribute. The levels for brand are randomized once per respondent, and then the product concepts are displayed in that new order for all choice tasks for that respondent. For example, respondent #1 may see brands in the order 2, 3, 1 for the three concepts in the task (left, middle, right), for all tasks. Then, respondent #2 may see the brands in the order 3, 2, 1 for the concepts in his questionnaire.
This strategy helps control order bias, without making respondents reorient themselves to new positions for, say, brands in each new task.
Natural Level Order. With this option, the concepts are shown in the natural level order for a selected attribute, such as "brand." Brand 1 will always appear in the first concept, brand 2 in the second concept, etc.--assuming that the researcher uses a Complete Enumeration or Shortcut design strategy (which enforces minimal overlap), has no prohibitions and shows as many concepts on the screen as there are brand levels in the study.
This strategy is often used in "shelf facing" displays (Advanced Design Module only), or in cases in which the researcher doesn't want to make respondents reorient themselves to new positions for, say, brands in each new task. However, using a fixed concept order may introduce order biases.
Show Levels of First Attribute no more than Once per Task
Sometimes with attributes like brand, the researcher wants each brand to appear exactly once per choice task. If you have 4 brands and are showing 4 concepts per task, this happens naturally with Complete Enumeration or Shortcut strategies--unless you have included prohibitions in your study. When prohibitions are in place, it is possible for a brand to be repeated within the same choice task. To avoid this possibility, you can check this box and the designer will ensure that levels of the first attribute appear just once per task. (Of course, this isn't feasible if you are trying to show more concepts per task than there are levels in the first attribute.)
Generate Design
When you click the Generate Design button, CBC automatically creates a design file using your current CBC settings that you upload to the server (for web-based data collection), or install on laptops/devices (for CAPI-based collection), or that is used in generating questionnaire files for paper-and-pencil collection. It also produces a test design report, letting you know whether the design can efficiently estimate the main effect utilities pertinent to your study. You should pay close attention to the design report to make sure the design efficiencies for the attribute levels in your study are acceptable and that the report does not state "Design Deficient" or have any asterisks in place of the estimated standard errors.
Please note that in some cases it may take a long time to generate a design if using the Complete Enumeration or Balanced Overlap methods. A typical study with few attributes and levels will face little delay (a few seconds) to generate hundreds of designs. CBC gives an estimate of the time remaining to generate the design. For testing purposes you may wish to generate few designs for the sake of speed, but once you are ready to post the survey to the Web (or use CAPI data collection) you may request that hundreds of designs be generated and saved to the design file.
Test Design Report Settings
Include Two-Way Frequencies in Test Design Report
Check this to include an additional analysis within the Test Design Report of joint frequencies of levels. These joint frequencies reflect the number of times each attribute level appeared with each other attribute level within the same product concept across all concepts and tasks of your design.
Import/Export Design
Some users may choose to alter CBC's designs, or use their own design instead. Use the Export/Import Design capability when performing such advanced options.