Introduction
The CVA/HB Module is an add-on component within Lighthouse Studio that uses hierarchical Bayes (HB) to estimate part-worths for ratings-based full-profile conjoint analysis (CVA) studies. HB considers each individual to be a sample from a population of similar individuals, and "borrows" information from the population in estimating part-worths for each respondent. With HB, CVA users can often achieve equivalent results (relative to OLS) using fewer tasks per person and/or fewer total respondents. Precisely how much improvement HB estimation offers over the standard estimation techniques depends on the project.
If using the CVA/HB module, it is possible to obtain good part-worth utility estimates even though respondents have evaluated a random subset of the questions within a questionnaire version (even fewer questions than parameters to be estimated). This approach can significantly reduce the number of questions each respondent is asked. However, we recommend you not do this unless sample sizes are quite robust.
HB estimation takes considerably longer than OLS or Monotone regression. Computation time will usually vary from about 3 to 20 minutes for most CVA data sets.
Although the CVA/HB module will produce results for very small sample sizes, we caution against using it in those instances. CVA/HB will produce results even with data from a single respondent, but with very small sample sizes it will have difficulty distinguishing between heterogeneity and error. How many respondents are required for robust HB estimation depends on the study design and the nature of the sample. We have seen HB perform well with samples as small as 80 respondents for ratings-based conjoint studies. HB may perform consistently well with even smaller sample sizes, though we know of no series of studies to substantiate that claim.
In any case, we suggest not using HB blindly. It is prudent to design holdout choices within your CVA questionnaires so that you can assess the performance of alternative part-worth estimation methods.
Background
The first traditional conjoint analysis applications in the early- to mid-1970s used non-metric estimation or OLS to derive part-worths. These techniques served the industry well over the first few decades of conjoint analysis practice. Even so, conjoint researchers have always faced a degrees of freedom problem. We usually find ourselves estimating many parameters (part-worths) at the individual level from relatively few observations (conjoint questions). It is often challenging to get respondents to complete many conjoint tasks, so researchers may sacrifice precision in the part-worth estimates by reducing the number of conjoint profiles. It is precisely in those cases that HB can be most useful.
HB became available in about the mid 1990s to marketing researchers. HB significantly improves part-worth estimates and produces robust results when there are very few or even no degrees of freedom. Several articles (see for example Lenk, et al. 1996 and Allenby, et al.1998) have shown that hierarchical Bayes can do a creditable job of estimating individual parameters even when there are more parameters than observations per individual.
It is possible using CVA/HB to estimate useful part-worths for an individual even though that respondent has answered fewer tasks than parameters to estimate. This can occur if respondents quit the survey early. However, the researcher may choose this approach by design. The researcher might randomly assign respondents a subset of a larger CVA design, so that across all respondents each task has roughly equal representation.
The CVA/HB Module estimates a hierarchical random coefficients model using a Monte Carlo Markov Chain algorithm. In the material that follows we describe the hierarchical model and the Bayesian estimation process. It is not necessary to understand the statistics of HB estimation to use this module effectively. The defaults we have provided make it simple for researchers who may not understand the statistics behind HB to run the module with consistently good results.
We at Sawtooth Software are not experts in Bayesian data analysis. In producing this software we have been helped by several sources listed in the References. We have benefited particularly from the materials provided by Professor Greg Allenby in connection with his tutorials at the American Marketing Association's Advanced Research Techniques Forum.
The Basic Idea behind HB
CVA/HB uses Bayes methods to estimate the parameters of a randomized coefficients regression model. In this section we provide a non-technical description of the underlying model and the algorithm used for estimation.
The model underlying CVA/HB is called "hierarchical" because it has two levels. At the upper level, respondents are considered as members of a population of similar individuals. Their part-worths are assumed to have a multivariate normal distribution described by a vector of means and a matrix of variances and covariances.
At the lower level, each individual's part-worths are assumed to be related to his ratings of the overall product profiles within the conjoint survey by a linear regression model. That is to say, when deciding on his preference for a product profile, he is assumed to consider the various attribute levels that compose that product, and add the value of each level to come up with an overall rating for the product concept. Discrepancies between actual and predicted ratings are assumed to be distributed normally and independently of one another.
Suppose there are N individuals, each of whom has rated conjoint profiles on n attribute levels. If we were to do ordinary regression analysis separately for each respondent, we would be estimating N*n part-worths. With the hierarchical model we estimate those same N*n part-worths, and we further estimate n mean part-worths for the population as well as an n x n matrix of variances and covariances for the distribution of individuals' part-worths. Because the hierarchical model requires that we estimate a larger number of parameters, one might expect it would work less well than ordinary regression analysis. However, because each individual is assumed to be drawn from a population of similar individuals, information can be "borrowed" from other individuals in estimating parameters for each one, with the result that estimation is usually enhanced.
For more information, please read the CVA/HB Technical Paper, available for download from our technical papers library at www.sawtoothsoftware.com.