Have an idea?

Visit Sawtooth Software Feedback to share your ideas on how we can improve our products.

How do I calculate the standard deviations of average utilities out of individual standard deviations?

Hello,
I was wondering how I can use individual standard deviations to get to the standard deviations (of average utilities) mentioned in the summary file.
My issue is that, if I take a look at the summary report, I can see that standard deviatios are decreasing for all attributes from one conjoint analysis to another conjoint analysis (where I added 2 attributes).
But if I take a look at the individual standard deviations, I can see that they are  (on average or when I sum them up) increasing from conjoint 1 to conjoint 2.
For me that is very contradicting.
I'm very grateful, if you could help me with that.
asked Jun 4, 2016 by Lisa
Could you please give more information about what software you are using, what utility estimation approach you are using, and which report(s) you are looking at?  That will help.
Oh sorry, should have been more specific.
I cannot see which SSI Web Version I was using because I only have the license at the university...
But I am using a HB approach and I was looking at the following output:
"hb_report" which gives you the average utilities of the whole sample and the standard deviations (together with importance etc.) and individual standard deviations in the file "hb_stddev".
Again, I am comparing two conjoint analyses in terms of change in standard deviations. For the standard deviations on the "population-level" I could see that they are decreasing from one conjoint analysis to the other, while the individual standard deviations are increasing from one to the other.
My supervisor told me now, that one should treat the average values from the summary report (at the "population level") with caution as the overall distribution consists of several normal distributions. The individual standard deviations would be more meaningful.
Would you also agree with that statement?
Thank you very much in advance.

1 Answer

+1 vote
Aha, thanks for the additional information!

Let's define a few things to make sure we're on the same page.

The HB Summary output will show you the "Average Utilities" and the "Standard Deviation" for the population.  This report is based on using zero-centered diffs (where the scale has been normalized so each respondent has similar scale...and this scaling is MUCH larger in magnitude than the raw HB utilities).  The Standard Deviation shown here is based on each individual's point estimates (the collapsed draws for each respondent...where we collapse them by taking their mean).  These standard deviations are estimated in the classical (Frequentist) way, based on point estimates that ignore the within-respondent variability (not proper Bayesian estimates!).

The "Hb_stddev" file gives you the estimate of standard deviations for each respondent and for each attribute level.  These are estimated from the raw utilities and using the individual respondent draws (proper Bayesian approach!).  You can see that these standard deviations will be quite different from the reported population standard deviation that has been estimated in the classical way.  #1 they are based on raw utilities rather than zero-centered diffs (and are thus on a different scale), and #2 they are individual level measures and are based on the uncertainty within each respondent as recorded in the draws.

To obtain a more appropriate Bayesian calculation of the standard deviations across the population for the attribute levels (on the raw utility scaling), one should compute those from the "alpha draws" (the mean population estimates from each iteration of the MCMC algorithm...only paying attention to the draws of population means after convergence has been assumed).  You could open the "alpha draws" file in Excel and compute the standard deviation of the columns (disregarding the first few thousand iterations prior to convergence).
answered Jun 9, 2016 by Bryan Orme Platinum Sawtooth Software, Inc. (178,915 points)
Hello Bryan,

Could you possibly explain where I find the file 'Hb_stddev'? I am facing a similar issue since I am looking for the actual individual standard deviations for each participant for each level. The current output of the HB analysis in Lighthouse Studio only provides a summary as well as the individual parameter estimates. Your support is highly appreciated!

Thank you,
Laura
...