A Mark-Recapture Approach to Estimating Item Pool Compromise

Area of Science:

• Psychometrics
• Ecological Research Methods

Background:

• Testing organizations must identify compromised exam materials to maintain assessment validity.
• Estimating the scale of item compromise is crucial for operational decisions regarding test forms and item pools.

Purpose of the Study:

• To introduce and exemplify the mark-recapture technique for estimating the population of compromised test items.
• To demonstrate the applicability of mark-recapture methods in large-scale testing programs for test security.

Main Methods:

• Adaptation of the mark-recapture technique, commonly used in ecological research, for psychometric applications.
• Demonstration through a conceptual example and a more realistic scenario relevant to testing programs.

Main Results:

• The mark-recapture technique provides a viable method for estimating the population of compromised items.
• Longitudinal tracking of estimated compromised item populations can inform test security strategies.

Conclusions:

• Mark-recapture offers a novel approach to evaluate the extent of item compromise in testing organizations.
• This method can aid in developing robust procedures for mitigating and managing compromised assessment materials.