Identifying person misfit using the person backward stepwise reliability curve (PBRC)

Area of Science:

• Psychometrics
• Educational Measurement
• Psychological Testing

Background:

• Aberrant response patterns can significantly impact the validity of psychometric assessments.
• Existing methods for detecting aberrant responding have limitations in visualization and identification.
• The Cronbach-Mesbach curve provides a theoretical basis for understanding response consistency.

Purpose of the Study:

• To propose a novel visualization technique for aberrant response patterns.
• To develop a person reliability index for identifying inconsistent responders.
• To assess the effectiveness of the proposed method against established indices.

Main Methods:

• Developed a person reliability index using the Kuder-Richardson 20 (KR-20) formula.
• Employed a backward stepwise deletion procedure to identify unreliable individuals.
• Utilized Person Response Curves (PRCs) to visualize the relationship between item difficulty and person skill.
• Compared the proposed method's results with established aberrant responding indices (Ht, U3).

Main Results:

• The proposed method successfully identified aberrant responders, with 100% concordance with Ht and U3 indices.
• Visualizations using Person Response Curves (PRCs) confirmed a lack of monotonicity in item difficulty and person skill for aberrant responders.
• The developed person reliability index effectively flagged individuals not contributing to the measurement of the latent trait.

Conclusions:

• The proposed visualization technique is a useful tool for identifying aberrant responders in psychometric assessments.
• The methodology enhances the detection of individuals whose response patterns compromise the measurement of a unified latent trait.
• This approach offers a more intuitive understanding of response inconsistencies and their impact on test validity.