Design Tab (ACBC)

For general information about ACBC interview setup, please refer to the section entitled: Sections and Flow for Adaptive CBC.  To learn about how to drop different sections in the ACBC questionnaire, visit: Skipping Sections in ACBC.

 

The key decisions to make when designing your ACBC study are:

 

How many total product concepts (cards) will each respondent evaluate?  This is equal to the Number of Screening Tasks x Number of Concepts per Screening Task.  We generally recommend asking each respondent to evaluate between 18 to 40 product concepts, depending on the attribute list length and complexity (see recommendations in the table below).  A good rule of thumb is to include enough cards so that each level taken forward to the ACBC survey appears at least twice, and preferably three times per respondent (you can check this quickly for different questionnaire settings using the Test Design... feature that generates dummy respondent data and tabulates the number of times each level appears across the product concepts).

 

How many attributes to vary from the BYO selections when generating product concepts (cards)?  At the beginning of the interview, respondents indicate which levels are "best" in the product they'd buy.  We call this configured product the "BYO Concept."  When generating new product concepts for respondents to evaluate, we choose near-neighbors to the BYO Concept, by varying a subset of the attributes that have been included in the BYO Concept.  For example, assume that there are 8 total (non-price) attributes in your study, and all were included in the BYO question.  If you choose to vary from 2 to 4 of the attributes (Minimum Attributes to Vary = 2; Maximum Attributes to Vary = 4), a new near-neighbor product concept could be generated by varying 3 of the attributes from their BYO-selected levels and retaining the other 5 attributes at their BYO-selected levels.  This idea is illustrated below.  To generate a near-neighbor concept, attributes 3, 4, and 7 have been selected to be varied from their BYO-selected levels.  The changed levels are indicated in green, italics:

 

 

The software will not allow you to vary more than half (+1) of the attributes included/chosen in the BYO exercise.  (This is an important feature for advanced questionnaires that use constructed lists or alternative-specific designs.  Even if the number of attributes used in the BYO-selected concept is different, the ACBC software will never vary more than half (+1) of the attributes from the BYO-selected levels when generating new product concepts.)

 

How many Unacceptables and Must-Haves questions to ask? The choice of how many Unacceptables and Must Haves questions largely depends on how many Screening Tasks you decide to display.  Please refer to Sections and Flow for Adaptive CBC for illustrations of the typical layout and how the Unacceptables and Must Haves are positioned within an ACBC study.  Generally, we suggest asking the first Unacceptables question after three Screening Tasks have been shown.  Then, we recommend showing the first Must Haves question after the second Unacceptables question.  This leads to the easy rule of thumb:

 

# Unacceptables = (# Screening Tasks - 3)

# Must-Haves = (# Screening Tasks - 4)

 

The more Unacceptable and Must-Haves questions that are asked, the larger the percent of levels respondents mark as Unacceptables will result.  One can increase the number of Unacceptables and Must-Have questions by building a larger pool of product concepts (cards), asking more Screener questions, and by showing fewer concepts at a time in each Screener question.

 

How many concepts should be brought into the Choice Tournament?  The choice tournament shows either two or three product concepts at a time (pairs or triples) and not-selected concepts are eliminated until an overall winner remains.  It takes t/2 such choice tasks to identify an overall winner if showing triples (in the case that t is odd and t/2 is not an integer, one rounds down to determine the required number of triples), and t-1 tasks if showing pairs.  Thus, if a respondent takes all 40 product concepts into the choice tournament (all 40 are marked "a possibility" in the Screening section), then 20 Choice Tournament tasks will be asked if shown in triples, or 39 if shown in pairs.  We generally recommend that you limit the number of concepts taken into the choice tournament to a little over half of your total product concepts (cards) and that you show the tasks in triples.  Thus, if you plan to show respondents 32 total product concepts, we'd recommend bringing up to about 20 products into the tournament (leading to a maximum of 20/2 = 10 choice tasks in that section).  Of course, if respondents mark fewer product concepts as "possibilities" in the Screener section, then fewer choice tasks will be needed to identify the overall winner.

 

Avoiding Dominated Concepts A dominated concept is a product alternative that is logically inferior to another concept (based on the Preference Order settings the researcher specifies).  Researchers have found that avoiding dominated concepts in conjoint designs adds only a very little to the quality of the final utility results.  However, it avoids the situation which sometimes bothers clients and respondents, where an obvious question is asked that might make that person doubt the logic behind the questionnaire.  Some degree of level balance and orthogonality is typically lost when avoiding dominated concepts (since it is just another form of prohibition that hinders the experimental design).  Depending on the study setup, it could be a relatively minor or a major effect upon the design quality.  The Test Design routine can help you assess the impact of avoiding dominated concepts on the precision of utility estimates. Note: if the experimental designer cannot find a design within the allotted time that satisfies the constraint of avoiding dominated concepts, it may need to show a dominated concept to produce a valid questionnaire for the respondent.  If this occurs, a message is stored in the design log table of the database.  You can download this from the Admin Module, by clicking Survey Administration | Advanced | Design Log Table.  This downloads a .csv file that you can view in a program such as Excel, showing per respondent messages related to the designer process.

 

Including the BYO-Selected Concept in the Choice Tournament Here, you specify whether the respondent's BYO-selected "ideal" product should be carried forward into the Choice Tournament.  For situations involving Summed Price or when not all attributes are shown in the BYO question, this can make a lot of sense (and lead to a richer Choice Tournament).  But, for cases in which all attributes are shown in the BYO question and summed pricing is not involved, the BYO concept should dominate any other concept that the experimental designer could generate, so it may not be advisable to carry the BYO concept forward to the choice tournament.  

 

---

Recommendations

 

Based on our experience with ACBC, we can offer some recommendations regarding questionnaire design settings.  Most studies will probably bring all attributes and levels into the ACBC section, so the table below reflects the idea that each person has an identically-sized design.  

 

We recommend that no more than about 7 total levels per attribute be brought into the ACBC part of the survey (other levels could be dropped as "undesirable" or "not available" using prior questions and constructed list logic).  We also recommend that no more than about 10 attributes be brought into the ACBC part of the survey (irrelevant attributes with virtually no importance could be dropped using prior questions and constructed list logic).

 

Depending on the number of attributes in the ACBC section of the survey, we recommend you consider the following Design settings (as approximate guidelines):

 

 

*Only if estimation of a new "None" parameter using Purchase Likelihood intent scale is required.  A "None" parameter automatically results from the Screening Tasks.

 

Note: if purely individual-level analysis is required (monotone regression) and/or sample sizes are quite small (n<50), we recommend increasing the Number of Screening Tasks by about 2 in each case (and also increasing the Maximum Number of Concepts in Choice Tournament by 4 in each case).

 

 

---

Test Design

 

 

This functionality lets you generate multiple test (dummy) respondents who answer the ACBC questions randomly.  See the Design Tab--Test Design topic for more information.