The problem of measurement bias in comparing selected subgroups

Area of Science:

• Psychometrics
• Statistical validation
• Subgroup analysis

Background:

• Subgroup difference estimates are crucial for validation and adverse impact evaluation.
• Direct range restriction introduces measurement bias, affecting mean estimates.
• This bias complicates subgroup comparisons, especially with unequal selection ratios or variances.

Purpose of the Study:

• To investigate measurement bias in subgroup mean estimation under direct range restriction.
• To develop corrected estimates for mean differences and Cohen's d*.
• To examine the presence of measurement bias under indirect range restriction.

Main Methods:

• Mathematical derivation of bias under direct range restriction.
• Formulation of corrected estimators for subgroup mean differences.
• Analysis of bias under indirect range restriction.

Main Results:

• Measurement bias is a function of selection ratio, reliability, and variance under direct range restriction.
• A corrected estimate for the mean difference and Cohen's d* is presented.
• Measurement bias is absent under indirect range restriction for selected means estimating selected true scores.

Conclusions:

• Direct range restriction biases subgroup mean estimates, necessitating corrected approaches.
• Corrected estimates improve the validity of subgroup comparisons in selection contexts.
• Indirect range restriction avoids measurement bias in selected mean estimates of true scores.