Recently we auto-populated a part of the BYO section based on prior knowledge from the survey questions and common sense.
The HB estimation using the BYO task resulted in the following scale factors:
BYO: 37582.014 (!!)
I'm not sure what to make of this as I do not understand the underlying math of the method. I suspect that this could indicate a problem of "non-identification" for level effects (with no observations) where auto-populated BYO preferences are uniform across the entire sample in the section. At the very least it isn't surprising that homogeneous BYO-data would generate extreme scale.
The following validation with hitrates etc didn't indicate anything was broken and we opted to keep "Otter's" factors active.
Are there any other interpretations? For example: Does the scale factor indicate that BYO section barely influenced the overall results or the opposite?
Are there other things to consider when auto-population of BYO?
Do homogeneous BYO preferences generally lead to larger scale factors?
Are there known pitfalls where such homogeneity in BYO-responses can "break" the results or unduly increase/reduce the role of BYO responses in ACBC estimation?
Thanks for your thoughts!