After you have created a design (either through CVA's designer or by importing a design from a text file), you can review the D-efficiency and receive additional diagnostic information by using CVA's Test Design functionality. When you select this option, a report is shown on the screen:
CVA Design Efficiency Test
Copyright Sawtooth Software
Tasks are 'Single Concept' using a seed of 1.
Based on 10 version(s).
Includes 140 total tasks (14 per version).
Each task includes 3 attributes.
Design Version Efficiencies
-------------------------------------------------------------
Version Efficiency
1 0.97528
2 0.97528
3 0.97528
4 0.97528
5 0.97528
6 0.97528
7 0.97528
8 0.97528
9 0.97528
10 0.97528
Attribute Efficiency Coefficients (including all versions)
-------------------------------------------------------------
Attribute Efficiency
1 0.999
2 0.995
3 0.999
One-Way Frequencies (by version)
-------------------------------------------------------------
Level Total Ver1 Ver2 Ver3 Ver4 Ver5 Ver6 Ver7 Ver8 Ver9 Ver10
1 70 7 7 7 7 7 7 7 7 7 7
2 70 7 7 7 7 7 7 7 7 7 7
1 46 4 4 4 6 6 6 4 4 4 4
2 48 4 4 4 4 4 4 6 6 6 6
3 46 6 6 6 4 4 4 4 4 4 4
1 35 3 4 3 3 4 4 3 4 4 3
2 35 4 4 3 4 4 3 3 3 4 3
3 35 3 3 4 3 3 4 4 4 3 4
4 35 4 3 4 4 3 3 4 3 3 4
Two-Way Frequencies (all versions)
-------------------------------------------------------------
Att/Lev 1/1 1/2 2/1 2/2 2/3 3/1 3/2 3/3 3/4
1/1 70 0 23 24 23 17 18 18 17
1/2 0 70 23 24 23 18 17 17 18
2/1 23 23 46 0 0 12 12 11 11
2/2 24 24 0 48 0 12 11 13 12
2/3 23 23 0 0 46 11 12 11 12
3/1 17 18 12 12 11 35 0 0 0
3/2 18 17 12 11 12 0 35 0 0
3/3 18 17 11 13 11 0 0 35 0
3/4 17 18 11 12 12 0 0 0 35
The Test Design report shows you the D-efficiency of each design that was generated (CVA generates 10 design versions by default), which provides an indication of the overall quality of each design. The highest possible number is 1.0, and the lowest is zero. For more information about D-efficiency, see Technical Details on the CVA Designer.
Next, CVA reports attribute efficiencies (described in greater detail below), assuming all designs are appended as an aggregate design plan. Low efficiencies for some attributes can be an indication that some levels of that attribute might not be included enough times in the design, or that this attribute has too many prohibitions (and is thus correlated with another attribute(s)). These coefficients can be useful diagnostics for a conjoint questionnaire design during the pilot stages of a study. If the relative efficiency is too low for a particular attribute, then additional questions should be introduced, or the design should be modified to have less dependence among the attribute levels.
If you have low D-efficiency for some or all designs, or if some attributes have particularly low efficiencies, you should consider removing constraints, asking more questions, or regenerating your design. Poor efficiencies can compromise the quality of your utility estimates.
Next, the one-way frequencies are displayed, considering all designs together ("Total" column) and then for each design separately ("Ver 1" through "Ver 10" columns). The frequencies reflect the number of times each level occurs in the design. Following that are the joint frequencies, or the number of times two levels appear together within the same product concept, across all designs. Although not shown above (to conserve space), similar joint frequency tables follow for each separate design.
More Details on Attribute Efficiency Calculation
The Attribute Efficiency statistics describe the quality of the conjoint design. We are mainly interested in the precision of estimates of differences in utility among levels of the same attribute. The relative error variance of the difference is obtained from elements of the inverse of the correlation matrix among independent variables. The relative error variance of the difference between two utility estimates is obtained by summing those two diagonal elements of the inverse and subtracting twice the corresponding off-diagonal element. We compute the relative error variances of all within-attribute pairwise differences.
Suppose there were an orthogonal design for the same attributes and levels, and all the levels of a particular attribute appeared the same number of times. (Such a design may or may not actually exist.) Then we could compute similar relative error variances for that "ideal" design.
Our "Relative Efficiency" index is the ratio of the sum of theoretical relative error variances for such an ideal design, divided by corresponding sum of relative error variances for the actual design used. The overall relative efficiency index is reported, as is an index for each attribute. The best possible value is 1.000, although values that high may not actually be achievable for some combinations of numbers of attributes and levels, since the required orthogonal designs may not exist.