I would like to calculate the D-efficiency of different designs manually (e.g. with R). Unfortunately, I don't get the same results as with Lighthouse Studio.
How did I proceed? I used the design matrix, e.g. of a (3,4,3,4)-design with 12 choice tasks with 3 alternatives each and 4 blocks (versions). In total, the design matrix X has 144 rows and 4 columns. To take the effect coding into account, I extended the matrix to 10 columns (main effects only, i.e. 2+3+2+3 = 10). For example, factor 1 with level 1 became (1,0), level 2 (0,1) and level 3 (-1,-1). After that I multiplied the inverse of X (10x144) with X, so (X’X). The next step was calculating the determinant, so |(X’X)|. The last step consists of calculating the pth root, i.e. |(X'X)|^(1/p) with p=10.
When doing this I get a D-efficiency of 66.13658. But when I use Lighthouse Studio with 4 respondents, so one respondent for each version, the D-Efficiency is 14.60480.
I suspect that the difference could have something to do with the number of respondents and the number of parameters estimated.
The output of Lighthouse Studio shows 14 parameters instead of 10, since the base levels are also parameter estimates. Now I wonder whether Lighthouse Studio in my example calculates the 10th or 14th root? But even when calculating the 14th root the result is 19.96712.
Furthermore, my approach (probably) implicitly assumes that one respondent fills all 48 choice sets instead of 4 respondents with 12 choice sets each. But for overall D-efficiency, that should not make a difference. Is that right?
In “Kuhfeld et al. (1994) - Efficient Experimental Design with Marketing Research Applications” also the intra block efficiency is calculated. I assume that in this case every block has its own design matrix X, so in my case 4 design matrices?
I suppose that I’m missing the forest through the trees. Could you please tell me what I’m missing?
Thanks in advance!