Accounting for Measurement Invariance Violations in Careless Responding Detection in Intensive Longitudinal Data:...

Area of Science:

• Psychological measurement
• Quantitative psychology
• Data analysis

Background:

• Intensive longitudinal data (ILD) collection, such as experience sampling methodology, poses participant burdens, risking careless responding.
• Careless responding, including random responding, compromises data validity and research inferences.
• Confirmatory mixture models, specifically fully constrained latent Markov factor analysis (LMFA), have been proposed to detect careless responding in ILD.

Purpose of the Study:

• To evaluate flexible variants of LMFA—fully exploratory LMFA and partially constrained LMFA—for detecting careless responding.
• To assess the performance of these models in the presence of non-invariant attentive responses, a common limitation of the fully constrained LMFA.

Main Methods:

• Simulations were used to compare the performance of fully exploratory LMFA and partially constrained LMFA.
• The models were evaluated based on their ability to distinguish between careless and attentive responding under conditions of violated measurement invariance.

Main Results:

• The fully exploratory LMFA model demonstrated effectiveness in reliably detecting and interpreting various forms of careless responding.
• This model successfully accounted for violations of measurement invariance, a key limitation of the fully constrained approach.
• The partially constrained LMFA model showed limitations in accurately detecting careless responses.

Conclusions:

• Fully exploratory LMFA offers a robust solution for identifying careless responding in ILD, particularly when measurement invariance is violated.
• Partially constrained LMFA is less effective for detecting careless responding under these conditions.
• Further research is needed to understand the nuances of these models and their application.