When respondent weights are being used, we report the unweighted respondent sample size, the weighted sample size, and the effective sample size. Our calculations of standard errors when weights are in use apply the effective sample size.
We use effective sample size to adjust standard errors based on suggestions from David Lyon (specifically, tutorials on weighting that he has taught at some recent Sawtooth Software events). The effective sample size calculation Lyon refers to comes from Kish's approximate formula for computing sample size (dating back to the 1960s).
Kish’s Effective Sample Size
Where:
Wi refers to the weight for the ith respondent
Example:
Let's imagine we had just five respondents in our dataset, with the following weights:
Respondent ID |
Weight |
1 |
1.5 |
2 |
0.8 |
3 |
1.2 |
4 |
1.1 |
5 |
2.5 |
To compute the effective sample size:
We square the sum of the weights: (1.5+0.8+1.2+1.1+2.5)2 = 50.41
We sum the squares of the weights: (1.52+0.82+1.22+1.12+2.52) = 11.79
The effective sample size is 50.41 / 11.79 = 4.28
Due to the differential weights we've applied across respondents in this example, the effective sample size is 4.28 respondents rather than 5 respondents. If we used even more extreme weights across respondents (larger ratio differences between respondent weights), the smaller the effective sample size becomes.
Standard Error of Product Share when Respondents Weights Are in Use
Where:
a refers to product a
Pa,i refers to the share of preference for product a for the ith respondent
Wi refers to the respondent weight for the ith respondent
refers to the average respondent weight
ESS refers to the Effective Sample Size