A constrained factor mixture analysis model for consistent and inconsistent respondents to mixed-worded scales

Area of Science:

• Psychological measurement
• Quantitative psychology
• Survey methodology

Background:

• Mixed-worded scales, containing both positively and negatively worded items, necessitate careful respondent engagement.
• Inconsistent answering patterns among respondents are a common, yet often overlooked, issue with mixed-worded scales.
• This heterogeneity in response patterns may explain wording-related effects observed in such scales.

Purpose of the Study:

• To introduce and validate a constrained factor mixture analysis model to identify distinct respondent classes (consistent vs. inconsistent) in mixed-worded scales.
• To assess the performance of this new model against traditional confirmatory factor analysis.
• To explore the characteristics and prevalence of inconsistent respondents.

Main Methods:

• Development and application of a constrained factor mixture analysis model.
• Analysis of five datasets featuring mixed-worded attitude scales across Germany, Australia, and the U.S., including children and adolescents.
• Comparison of the proposed model with confirmatory factor analysis incorporating a method factor.

Main Results:

• The constrained factor mixture analysis model demonstrated expected parameter patterns and outperformed traditional methods across all datasets.
• Between 7% and 20% of respondents were identified as belonging to inconsistent answering classes.
• Class membership correlated with respondent characteristics like reading achievement, cognitive skills, and conscientiousness.

Conclusions:

• The proposed factor mixture model effectively identifies and differentiates respondent classes in mixed-worded scales.
• A significant proportion of respondents exhibit inconsistent answering patterns, impacting scale validity.
• Findings have implications for improving the design and interpretation of mixed-worded scales in research and practice.