Standard Error Estimation for Subpopulation Non-invariance.

Area of Science:

• Psychometrics
• Educational Measurement
• Statistical Modeling

Background:

• Score linking is crucial for comparing assessment scores across different scales or conditions.
• Ensuring fairness requires the linking function to be invariant across diverse subpopulations.
• Existing methods struggle to accurately evaluate subpopulation differences due to complex error dependencies.

Purpose of the Study:

• To develop and validate novel statistical methods for assessing score linking invariance across subpopulations.
• To address the overestimation of standard errors in current approaches that ignore linking error dependencies.
• To enhance the power of detecting fairness violations in educational and psychological assessments.

Main Methods:

• Development of statistical models that explicitly incorporate linking error dependencies.
• Simulation studies to evaluate the accuracy and performance of the proposed methods under various conditions.
• Application of the new methods to a real-world dataset for practical validation.

Main Results:

• The proposed methods provide accurate standard error estimates for subpopulation differences in linked scores.
• Neglecting or misrepresenting linking error dependencies leads to substantial overestimation of standard errors.
• The new approach significantly increases statistical power to detect non-invariance across subpopulations.

Conclusions:

• Accurate accounting for linking error dependencies is essential for valid fairness evaluations in score linking.
• The developed methods offer a more reliable approach to ensuring equity in standardized testing.
• Improved standard error estimation facilitates more robust detection of fairness issues, supporting equitable assessment practices.