In a previous section entitled Coding the Information for Parameter Estimation we described the strategy for coding respondent answers into choices from sets of available options. We have employed HB effectively to estimate part-worth utilities for ACBC experiments. Details regarding our implementation of HB may be found in the CBC/HB product manual and in the CBC/HB Technical Paper.
Output Files
We display a summary report on the screen after estimation is completed, where the part-worth utilities are summarized as normalized zero-centered diffs. Importance scores are also displayed, along with the standard deviations of these measures.
The diffs method rescales utilities so that the total sum of the utility differences between the worst and best levels of each attribute across attributes (main effects) is equal to the number of attributes times 100. It removes the often strong differences in the magnitude of "scale" that make it hard to compare raw utilities across respondents or between groups of respondents.
The utility run report displayed to the screen has multiple tabs. The first tab includes the summary report as just described. The second tab contains the raw part-worth utilities per respondent (same information as saved to the .csv and .hbu files, described below). The third tab includes the normalized (zero-centered diffs) part-worth utilities, appropriate for subsequent segmentation/cluster analysis.
Various output files may be written or exported to the study directory:
Most Commonly Used Files:
.csv
Comma-separated values file containing the part-worth utilities (point estimates, one row per respondent). May be opened directly with Excel, and contains a header row containing labels. These are raw utilities, appropriate for market simulations. If using utilities for segmentation analysis (such as cluster analysis), one should use normalized utilities (e.g. zero-centered diffs) rather than raw utilities.
.hbu
Contains the respondent part-worth utilities (same as written to the .csv file). This file is formatted as text-only, and may be imported into our standalone market simulator for market simulations (note: only the utility run from an ACBC study is imported into SMRT. ACBC's .CHO file is incompatible with our market simulator and should not be imported). See also .hbu file format.
.sum
Contains final estimates of the population means and covariances.
Files Containing Diagnostics to Assess Convergence
.log
Contains a history of RLH, Avg. Variance, and Parameter RMS, which is useful information to assess convergence.
.alp
Contains a history of the estimated population mean for part-worths. One row is written per draw saved (file is only written if Save Draws is enabled).
.gamma
If using Otter's Method for estimating task-specific scale factors, this file contains the estimated scale factors for the three sections in ACBC used for part-worth estimation (BYO, Screeners, Choice Tasks). One row is written per draw saved (file is only written if Save Draws is enabled).
Files Containing Advanced Information
.cov
Contains the estimated variance-covariance matrix for the distribution of part-worths across respondents. Only the elements on-or-above the diagonal are saved. One row is written per draw saved (file is only written if Save Draws is enabled).
.dra
Contains estimated part-worths for each individual saved from each nth iteration, where n is the Skip Factor for Using Random Draws. The file is formatted like the .hbu file, without the header.
.std
Contains the within-respondent standard deviations among random draws. Only created if you elect not to save random draws.
Otter's Method
One consideration when estimating part-worth utilities from ACBC is the large difference in scale (response error) for the three main sections of the questionnaire. If using generic HB estimation, a key assumption is that the three types of tasks can be combined within the same dataset, even though we have observed that each section has different scale factor. Even though the BYO section has larger scale than the other two sections, we ignore this fact when using generic HB to estimate the parameters across the tasks.
We have wondered about the practical effect of the differences in scale for the three sections when using a generic HB model. We have been fortunate to benefit from the expertise of Dr. Thomas Otter, a recognized expert regarding HB application to choice experiments. Otter has built more sophisticated HB models that separately model scale for the three sections in ACBC questionnaires. He has found that the generic HB model performs very nearly as well as the more sophisticated models. You can turn on "Otter's Method" for accounting for differences in scale during HB estimation through the HB interface provided in ACBC.
When Otter's method is selected, an additional diagnostic file is written to disk when Save Random Draws is enabled that reports the relative gammas (scale factors). By default, the scale factor for the Screener section is constrained via very strong priors to remain around 1.0. Gammas for BYO, Screeners, and the Choice section are written to the .gamma file (in that order). A gamma of less than one indicates that this section of the survey has smaller scale than the Screener section. The final utility estimates are placed on the scale commensurate with the Screener section.
For more information, please see:
Otter, Thomas (2007), "HB Analysis for Multi-Format Adaptive CBC," Sawtooth Software Conference Proceedings, Sequim, WA.