Person-Fit as an Index of Inattentive Responding: A Comparison of Methods Using Polytomous Survey Data.

Area of Science:

• Psychometrics
• Quantitative Psychology
• Statistical Modeling

Background:

• Self-report measures are susceptible to response biases, such as inattentive or careless responding, which can compromise data integrity, especially in low-stakes assessments.
• Existing methods for detecting aberrant response patterns include a priori and post hoc approaches, with nonparametric person-fit statistics previously showing high accuracy for dichotomous outcomes.

Purpose of the Study:

• To evaluate the accuracy of parametric and nonparametric person-fit statistics in detecting inattentive responding on measures utilizing polytomous response scales.
• To assess the impact of using person-fit statistics for identifying inattentive respondents on confirmatory factor analysis (CFA) model fit.

Main Methods:

• Employed Receiver Operating Characteristic (ROC) analysis to determine the accuracy of various person-fit statistics in identifying a proxy for inattentive responding.
• Utilized confirmatory factor analysis (CFA) fit indices to examine how the application of person-fit statistics influences overall model fit.

Main Results:

• ROC analysis indicated that the nonparametric H^T statistic demonstrated the highest accuracy (largest area under the curve) in detecting inattentive responding.
• CFA fit indices revealed that the impact of employing person-fit statistics is contingent upon the specific purpose and chosen cutoff for identifying inattentive respondents.

Conclusions:

• The nonparametric H^T statistic is a highly accurate method for detecting inattentive responding on polytomous scales.
• The utility of person-fit statistics in data cleaning and analysis depends on the researcher's objectives and the criteria set for excluding participants.