Have an idea?

Visit Sawtooth Software Feedback to share your ideas on how we can improve our products.

Prioritizing level-balance, Prohibitions vs Alternative-Specific

I know that alternative-specific designs should be favoured, and that's what I tested at first.

However (in ACBC Test Design), the conditional level of the primary attribution seems to be undersampled compared to the other levels of that primary attribute. The number of choice tasks is sufficient such that level frequencies are sufficient, but its undersampling means that it's not shown in enough combinations with another key attribute.

e.g., Say 'a' is the conditional level for the PrimaryAttribute. I would want 'a' to be seen enough times with AttributeB's levels of 'b1', 'b2', 'b3', 'b4.'

In tallying how often concepts occur during manual run-throughs of 24 choice tasks, I would see 'a' being shown far fewer times than other PrimaryAttribute levels; further, combinations that appear tend to be 'a/b1', and no other combinations.  I suspect that there could be an interaction between PrimaryAttribute and AttributeB, so I don't think the lack of variety in combinations would give me good data. Please correct me if I am wrong!

So, I'm considering using Prohibitions. I know they should be used sparingly. Wondering if there are any guidelines for their use when considering the number of attributes and levels. The concept in mind for me is that prohibitions are bad because it reduces orthogonality (I think? or introduces correlated redundancies into the design).

Is there a guideline for prohibitions in asymmetric designs? By asymmetric, I mean imbalance in levels among attributes involves in the prohibitions.

Suppose that my design is 4 attributes (7 x 7 x 4 x 2). Suppose that "AttributeA" has 7 levels and "AttributeB" has 2.  If I set level 'a1' of "AttributeA" to be prohibited with level 'b1' of "AttributeB", can one guess the "danger level" of the prohibition based on number of levels?

If AttributeB has 2 levels, then the 2 levels will be sufficiently seen in many combinations with the other attributes. While a product with 'a1' will be 100% associated with the appearance of 'b1', a product with 'b1' doesn't always associate with the appearance of 'a1'.   Is that a valid way of thinking about it, or no?

That's the specific example that I have in mind so far. Any other heuristics or guidelines regarding setting prohibitions in an informed manner would be most appreciated (or other thoughts on my alt-specific issue)!

Side-note: This is an ACBC with no BYO and no screener for substantive reasons; it's essentially a CBC with constructed listed designed in ACBC module.
asked Feb 11, 2020 by yorkmr (435 points)

1 Answer

+1 vote
A few prohibitions may be used, as long as you run the Test Design to check the effect of the prohibitions on the utilities for the levels involved.  The Test Design report in ACBC gives you the standard errors assuming aggregate logit estimation when prohibitions are present and when they are not.  The nice thing about this Test Design procedure is that you do not need to run it twice (with and without prohibitions).  It will take an ACBC design where you have specified a few prohibitions and generate two sets of data automatically: one set with the prohibitions and one set assuming no prohibitions.  The task for you then is to compare the standard errors of the utilities with and without the prohibitions as shown in the Test Design report.

You are correct that introducing prohibitions will introduce some degree of correlation in the experimental design.  If that correlation is relatively low, then the results are not much harmed.

The relative efficiency of conjoint design A with no prohibitions specified and design B with prohibitions with respect to a specific attribute level (Say, Level5) is equal to the ratio of the squares of the standard errors.

For example,

SE5a^2 / SE5b^2

where SE5a is the Standard Error for level 5 from design A (without prohibitions) and SE5b is the Standard Error for level 5 from design B (with the prohibitions).

As a numeric example, let's say level 5 has a standard error from the ACBC Test Design aggregate logit report without prohibitions of 0.04 and level 5's standard error with prohibitions is 0.05.  The relative efficiency of the design without prohibitions compared to the design with prohibitions is equal to:

0.04^2 / 0.05^2 = .64

In other words, the effect of the prohibition is equal to 1-0.64 = 0.36, meaning a 36% loss in efficiency when the prohibition is in place, considering just his attribute level.  

You can explain to the client that imposing this prohibition has got the same negative effect on the precision as throwing away 36% of the sample.
answered Feb 14, 2020 by Bryan Orme Platinum Sawtooth Software, Inc. (195,490 points)