A Two-Step Method Based on lz* for Identifying Effortful Respondents

Area of Science:

• Educational Measurement
• Psychometrics
• Data Mining

Background:

• The likelihood-based person-fit statistic (lz*) is vital for detecting non-effortful respondents in educational assessments.
• lz* accuracy is compromised by biased item parameter estimates when non-effortful respondents are present.

Purpose of the Study:

• To develop a more accurate method for estimating item parameters for person-fit analysis.
• To enhance the precision of the lz* statistic in identifying non-effortful respondents.

Main Methods:

• A two-step approach was proposed, combining data mining with person-fit statistics.
• K-means clustering was utilized to identify distinct respondent groups (effortful vs. non-effortful).
• Item parameters were re-estimated using data solely from the identified effortful group.

Main Results:

• Item parameter estimates derived from the effortful group were found to be more accurate.
• The enhanced lz* statistic demonstrated improved precision in identifying non-effortful respondents, particularly under high non-effort severity.
• The proposed method effectively mitigates bias introduced by non-effortful respondents.

Conclusions:

• The data mining-enhanced, two-step method provides a robust approach to person-fit analysis.
• This technique improves the reliability of identifying respondents who are not exerting effort in educational assessments.
• Accurate item parameter estimation is crucial for the validity of person-fit statistics.