Should I set interaction effects when running LCA? and Analyse Logit analysis from main effects only?

Question

My steps are
1) Count Analysis: it's shown 1 pair of Joint effects as significant (P value p < .01)
2) Logit: done 2 models
1. with main effect only
2.with 1 pair joint effect from count
Results showed:
Log-likelihood joint effect =-3180.10927
main effect=    -3204.68454  
chi2=    49.15054531
df=9
from this: does this means for Logit analysis I will need to Analyse Logit analysis from main effects only?
3) LCA: from step 1 and 2 does this means.. Should I set interaction effects when running LCA or no?

Thank u so much!

Comments

If you are working inside of Lighthouse Studio, you can run the interaction search as one of your analysis types that will compare the main effect and interaction effect model and give you a % boost to the fit statistic.  If the number is very small, like < 1% then it probably isn't doing much to actually change the model parameters.

Are you planning on using HB in the end to produce individual-level models?  These usually outperform aggregate models since you aren't combining lots of different people together and often eliminate the need for interaction effects all together.  It's not uncommon to run Latent Class to look for hidden segments, but still use HB to summarize the groups and as the basis for your simulators.

---

Hi Brian,
I am working in Lighthouse Studio. I checked  interaction search and found out it is like running a logit model like I did;however, I calculate the  Log-likelihood  between the 2 models (with one interaction effect and with only main effect) as results here:
Log-likelihood joint effect =-3180.10927
main effect=    -3204.68454  
chi2=    49.15054531
df=9
p value chi2=  49 and df=9 is <0.01...so, it is significant.
Which model should i choose to intepret the result from logit method here? I think choose the main effect only one? :P
and No unfortunately, i will only be running LCA after Logit.
Should I set the LCA setting to be with 1 pair interaction effects or not Brian?

Thanks a lot :)

---

Right, but something that is statistically significant doesn't necessarily mean it's actually doing anything for the model (is it practically significant?), plus if you segment people you might not need the interaction effect anymore.  Unfortunately it's a bit more complicated than simply following a chi square test statistic to decide if you should actually use it or not.  What is your sample size? What is the percent gain in the fit from the interaction search?  If you run logit with and without the interactions, do any of the parameters change by very much, or are you ending up with a lot of very small interaction adjustments that don't really change anything?

Also why are you not wanting to use HB?  If you are interested in looking at groups of people, doesn't it make more sense to build individual models to try to capture respondent heterogeneity as much as possible instead of lumping them together into groups and calculating group-level models?

---

"practically significant" I am not so sure what you mean here XD  
But that's a great idea! I compared to result in logit and found out that : 1.the effect for each variables gives the same results interms of higest and lowest number for each attributes level (even some numbers are 0,3 difference)
2.The attribute important give difference result for
2,1 main effect only ( Ranking became :1.Battery(31.82) 2.Price(28.18))
2.2 1 joint effects: as it is a joint effect between Price and bettery ( Ranking became:1.Price(26.95) 2.Battery( 26.17)
From here, Which model should i choose to intepret logit Brain?
-sample size= 185 people
I see your point and agreed ! as this is only a report to the company from my study we are not doing HB in this case and the teacher told us to use that rule i told you; so i have not try out the test u suggest yet.
and for LCA should I use the model with 1 pair interaction effect as i understand that interaction effect should not be ignore since there is multiple independent variables(which are attributes: I have 6 in total) in this case right?

Thank uu!

---

Also, Brian
 from what you say  here>>if you segment people you might not need the interaction effect anymore.  

I wonder why for this case? Thank u !

---

Consider a simple example where you are looking in to types of cars (sports car versus large luxury sedan) and color of the car (red and black).  A sports car might pair very well with the color red, while red might be a very bad color for a big luxury car.  So, if you are building an aggregate model that includes people who like both types of cars, you need an interaction effect to correctly model why red gets chosen sometimes and black gets chosen other times (it depends on the car type).

But, if you build a model for each individual, it's much less likely that you need an interaction effect.  If someone is interested in a sports car, they will probably choose the color red more.  If they are interested in a large luxury car, they will choose red less, and sports cars and big luxury cars are not very good substitutes for each other.  No need for the interaction as we have modeling the individual preference.

I would still recommend that you use HB even if you are just reporting utilities and attribute importance.  Choosing interactions and how many segments in a Latent Class model is a mixture of art and science.  You can't just follow fit statistics, but have to think if it actually makes sense and improves the model.

---

Thank you for the great explanation Brian and I totally agree with you, I will do HB to confirm everything that I did if time allow.
I did come across this example on the manual; now u confirm my hypothesis which is awesome!
As I will have to explain why I add interaction in LCA,is there Papers to support this example, Brian?
So now, my LCA comes wiht interaction..Does this mean I will also have to pick the logit with interaction as well?
Ps, *****" is displayed in place of the standard error and t-ratio in logit for interaction model but not in logit of only main effects..Can you maybe explain why this is the case?
( my rationale for this would be: where it shown **** It was tying to estimate interaction effects among attributes involved in a prohibition, but then this should happen in the logit model with main effect as well, i wonder why this is not the case here!? )

Thank you sooo much for your help! :D

Answer 1

I don't have any papers to back that up, sorry.

If you are seeing asterisks it means your model is deficient and does not support the interaction effect.  You are correct that the software is not able to estimate an interaction if you have prohibitions between those attributes.

Comments

Hi Brian,
No worries.Thank you
I see! So, can i use this logit model to interpret the result  even there is  asterisks..Is this still good enough?

Also, asterisks is displayed in logit with interaction model but not in logit of only main effects..Can you maybe explain why this is the case?

Thank you for such a quick reply. I am writing my report now :D

---

If you are getting asterisks then the model is deficient and should not be used.  If you have prohibitions between attributes, you can't estimate an interaction between them and should stick with a main effects model.

---

Thank you Brian.
I have all effect values but some Std Error and t Ratio are missing.

To sum up, I will use main effect model for Logit and Interaction effect model for LCA. Will these be ok? and does all these reasons you told me are strong enough to back up why i did not use the same type of model for these 2 analyses?
Thank you so much!

---

I can't really comment on that. It doesn't make sense that latent class will run with an interaction but logit will not. You probably need to send your survey files in to support@sawtoothsoftware.com.  If you are on an academic subscription, though, you are supposed to go through your professor with technical questions first.

---

Thank You Brian. I will! and I will start writing the result of logic with main effect first then.
So, generally same type of analysis should be run on both model right?

My reason for using an interaction effect is that..the product is wireless headphones and when people look for high battery it will goes with higher price and it is not the case for company offering a high performance product with low price. Also, this attribute pair of battery and price is the main decision purchasing point amoung other attributes that I have from asking the participants also. Is this the logical reason like your car case?

and I did ran LCA with main effects CAIC and BIC is minimized at 6 groups. However, segment size is smaller than 10% ;thus, it does not past the criterria. In this case, can i pick any groups let's say from 2 or 3 or 4 groups that although gives higher number of CAIC and BIC as the number of group decrease but i am able to segment the segment perferene's clearly and the number of groups mathes the company product offering?

Thank you so much Brian!

---

We're really getting more into opinionated guidance as opposed to technical support. I can't advise you on cluster solutions or model specifications without digging into your survey and models, which goes beyond our support.  Our Latent Class technical paper at https://www.sawtoothsoftware.com/support/technical-papers/sawtooth-software-products/cbc-latent-class-technical-paper-2004 provides some guidance on choosing the number of segments with a sample computation: https://www.sawtoothsoftware.com/support/technical-papers/sawtooth-software-products/cbc-latent-class-technical-paper-2004