Combining Propensity Scores and Common Items for Test Score Equating

Area of Science:

• Statistics
• Educational Measurement
• Psychometrics

Background:

• Ensuring score comparability across test forms and groups is a key challenge in educational testing.
• Current methods for test score equating often rely on common items or assumptions of group similarity.
• Novel approaches are needed to enhance the fairness and accuracy of test score comparisons.

Purpose of the Study:

• To develop and evaluate a novel statistical method for test score equating.
• To combine propensity scores, based on background covariates, with common item information for improved score comparability.
• To assess the performance of this integrated method using empirical and simulation studies.

Main Methods:

• Utilized propensity scores derived from test taker background covariates.
• Integrated propensity scores with common item information using kernel smoothing techniques.
• Conducted an empirical analysis on a high-stakes college admissions test and a simulation study.

Main Results:

• The proposed method, integrating propensity scores and common item data, demonstrated reduced standard errors and bias compared to using either source alone.
• Balancing groups on test-taker covariates was shown to enhance the fairness and accuracy of score comparisons.
• The study highlighted the benefits of utilizing all available data for improved score comparability.

Conclusions:

• The novel approach effectively enhances test score comparability by integrating propensity scores and common item information.
• This method offers a more robust and accurate way to ensure fairness in educational testing across diverse groups.
• Considering all collected data, including background covariates, is crucial for improving the precision of test score equating.