I am struggling a bit with finding the right settings for my ACBC/HB estimation. I read quite a good number of forum posts regarding this but still feel unsure. Highly appreciate any input.

Background: Academic study. Holdout tasks not used.

1.) Prior degrees of freedom and prior variance.

In the paper "What Are the Optimal HB Priors Settings for CBC and MaxDiff Studies?" there is a recommended list of settings depending on sample size and # of attributes. Since that's a general recommendation, I am not quite sure whether I should apply those settings or keep the default ones? e.g. if having 8 attributes, I would take a prior variance of 0.3 which is quite different compared to the default. Changing prior variance and degrees of freedom affects the results (rel. importances and part-worth utilities) so I am quite cautious about which values I take for both.

2.) # of iterations

Are 20k+20k sufficient or should I change this setting?

3.) Estimate Task-Specific Scale Factors (Otter's Method) should be activated for ACBC, right?

4.) Internal validity.

In my ACBC, I do not intend to include holdout tasks to ensure that the survey does not get too overwhelming for respondents (feedback so far is that it is quite complex, so I'd better not give them additional burden with holdout tasks). This means I can't use hit rates or mean absolute error to tell how good the model is.

Taking into consideration that I don't use holdout tasks and that I don't have real-work reference figures, what can I use to state how good the internal validity is? There are factors like McFadden pseudo R2 , RLH and Chi-Square but I don't know how to interpret them.

In one study I read that the internal consistency was checked by how much BYO-selections coincided with the choice tournament's winining concept. Is that a good method? And what would be a threshold for acceptable consistency? Does e.g. an average 70% match between BYO concept and winning concept speak for a good consistency?

5.) How do I interpret Pct. Cert., RLH and Avg Variance?

I feel it is not sufficient to just say "RLH is >0.33" when having 3 concepts per choice task. What is the threshold to declare RLH good or not-so-good? Same for Pct. Cert and Avg Variance.

6.) Comparing results depending on different demographic groups

The "normal" procedure would be to export zero-centered values of all data and then run e.g. a t-test in SPSS, correct?

I am unsure because I read somewhere that demographic groups can also be taken directly into the HB estimation? That seems very complex for me.

4. There is no summed pricing in the study (no price attribute at all). So I guess it may make sense to compare BYO with winning concept?

As this fits the topic: In the Test Design feature of ACBC I can see that standard errors of all attribute levels are <0.05 for a particular sample size. In the ongoing study I have less respondents as simulated and I want to check how "far" I am from the targeted <0.05 mark for all levels. The approach would be to simply calculate standard errors of the attribute levels based on RAW data, not zero-centered, correct? I just want to check if I can end my survey with a smaller sample size so I am waiting to the point where all SEs are <0.05