Detecting non-content-based response styles in survey data: An application of mixture factor analysis

Area of Science:

• Psychological Measurement
• Data Quality Assurance
• Statistical Modeling

Background:

• Self-report surveys are susceptible to careless or inattentive responses.
• Non-content-based (nCB) responses compromise data integrity, leading to biased analyses and score misinterpretations.

Purpose of the Study:

• To propose and evaluate a factor mixture analysis (FMA) model for detecting nCB responses.
• To assess the effectiveness of FMA in identifying problematic survey responses across different data conditions.

Main Methods:

• Factor mixture analysis (FMA) was specified and tested.
• Simulated data (Study 1) and real-world survey data (Study 2) were used to evaluate FMA performance.
• Sensitivity and specificity of FMA were calculated for various scale types.

Main Results:

• FMA demonstrated robust sensitivity (0.60-0.86) and excellent specificity (0.96-0.99) on mixed-worded scales.
• FMA performance was suboptimal on scales comprising only positive items due to acquiescence.
• In real data, FMA identified 6.5% of cases with anomalous patterns, improving model fit upon removal.

Conclusions:

• FMA is a valuable tool for detecting non-content-based responses, particularly on mixed-worded scales.
• Care must be taken when applying FMA to scales with only positive items.
• Removing detected nCB responses significantly enhances the quality and interpretability of survey data.