Calculating Bias in Test Score Equating in a NEAT Design

Area of Science:

• Educational Measurement
• Psychometrics
• Statistical Analysis

Background:

• Test score equating ensures comparability across different test forms, especially with non-equivalent groups.
• The non-equivalent group with anchor test (NEAT) design is a common practical approach.
• Evaluating bias in equating methods is critical for score interpretation.

Purpose of the Study:

• To compare bias amounts under various conditions using chained equating versus frequency estimation.
• To investigate the influence of five different criterion functions on bias calculations.
• To assess how factors like group ability differences and test form characteristics affect equating bias.

Main Methods:

• Utilized real data from a college admissions test and simulated data.
• Employed chained equating and frequency estimation with identity, linear, equipercentile, chained, and frequency estimation criterion functions.
• Examined bias under conditions of varying group ability, item difficulty, test form length, correlations, and sample sizes.

Main Results:

• The choice of criterion function significantly influences the evaluation of bias in equating methods.
• Empirical and simulated data demonstrated that different conditions highlight varying levels of bias.
• Bias definition critically affects the preference for specific equating methods in different scenarios.

Conclusions:

• The definition of bias is paramount when selecting equating methods for standardized tests.
• Practical implications for standardized testing include careful consideration of criterion functions.
• Recommendations are provided for calculating bias to evaluate equating transformations effectively.