A Simple Approach for Differential Test Functioning Based on Sum Scores.

Area of Science:

• Psychometrics
• Educational Measurement
• Quantitative Psychology

Background:

• Differential Test Functioning (DTF) assesses group differences in test scores beyond latent trait variations.
• Current DTF effect size measures often rely on complex Item Response Theory (IRT) models.
• Raw score interpretation of DTF magnitude is highly desirable for practical applications.

Purpose of the Study:

• To propose and evaluate a novel observed-score-based approach for estimating DTF magnitude directly in raw score points.
• To introduce Index S and its standardized version (Index S_std) as accessible measures of DTF.
• To demonstrate the utility of the proposed indices through simulation studies and an empirical application.

Main Methods:

• Developed Index S, which stratifies examinees by a sum score from non-DIF items.
• Summarized within-stratum mean score differences and aggregated them using observed distribution weights.
• Employed two-parameter logistic (2PL) simulation studies with varying parameters (sample size, DIF type, etc.).
• Assessed estimation accuracy (Bias, RMSE) and correlation with IRT-based DTF indices.

Main Results:

• The proposed Index S accurately estimates DTF magnitude in raw score points.
• Simulation studies demonstrated good estimation accuracy (low Bias and RMSE) across various conditions.
• Index S showed strong correlations with established IRT-based DTF measures.
• The empirical application provided accessible raw-score interpretations of DTF.

Conclusions:

• Index S offers a simple, interpretable, and accurate method for quantifying DTF in raw score units.
• This approach enhances the practical utility of DTF analysis for both psychometricians and non-specialists.
• The proposed indices provide valuable insights into group differences in test performance, complementing existing IRT methods.