An Iterative Scale Purification Procedure on l_z for the Detection of Aberrant Responses

Area of Science:

• Psychometrics
• Educational Measurement
• Statistical Modeling

Background:

• Aberrant response behaviors, including cheating and guessing, can compromise the validity of test results.
• Person-fit statistics, such as the widely used lz index, are employed to detect these behaviors.
• The accuracy of person-fit statistics relies heavily on precise estimation of item and person parameters, which can be negatively impacted by aberrant behaviors.

Purpose of the Study:

• To develop and evaluate an iterative procedure for obtaining more accurate person parameter estimates.
• To enhance the performance of the lz statistic in detecting aberrant response behaviors.
• To assess the effectiveness of the proposed iterative method under various conditions.

Main Methods:

• An iterative procedure was developed to refine person parameter estimates.
• Simulations were conducted to compare the iterative procedure with a non-iterative approach.
• Evaluations were performed under conditions of known and unknown item parameters and three types of aberrant responses: difficulty-sharing cheating, random-sharing cheating, and random guessing.

Main Results:

• The iterative procedure demonstrated superior performance compared to the non-iterative method.
• The proposed procedure effectively controlled Type-I error rates.
• Improved statistical power for detecting aberrant responses was observed with the iterative method.

Conclusions:

• The developed iterative procedure enhances the accuracy of person parameter estimation.
• This leads to improved performance of the lz statistic in identifying aberrant response behaviors.
• The method shows practical utility, as demonstrated by its application to a high-stakes intelligence test.