Differential Item Functioning Effect Size From the Multigroup Confirmatory Factor Analysis for a Meta-Analysis: A...

Area of Science:

• Psychometrics
• Educational Measurement
• Statistical Analysis

Background:

• Differential item functioning (DIF) is crucial for ensuring test fairness across different groups.
• Synthesizing DIF effect sizes across studies is challenging but necessary for robust conclusions.
• Existing methods for DIF effect size synthesis have limitations.

Purpose of the Study:

• To propose and evaluate a novel meta-analytic approach for synthesizing differential item functioning (DIF) effect sizes.
• To examine the performance of this new approach under various simulation conditions.
• To provide a practical method for summarizing DIF magnitudes across multiple studies.

Main Methods:

• Developed a meta-analytic summary of multigroup confirmatory factor analysis (MGCFA) effect sizes (MGCFA-ES).
• Conducted a Monte Carlo simulation with 108 conditions varying item difficulty, DIF magnitude, sample size, and correlation matrix type (tetrachoric, adjusted Pearson, Pearson).
• Assessed performance based on bias, mean square error, confidence interval coverage, standard errors, Type I error rates, and statistical power.

Main Results:

• The meta-analytic summary of MGCFA-ES performed best when using tetrachoric correlation matrices, showing favorable bias, MSE, CI coverage, and power.
• Performance was also reasonable with adjusted Pearson correlation matrices.
• Optimal performance was observed with tetrachoric matrices under conditions of high item difficulty, large DIF, and large sample sizes.

Conclusions:

• The proposed meta-analytic approach offers a viable option for synthesizing DIF effect sizes across studies.
• Using tetrachoric correlation matrices with MGCFA is recommended for accurate DIF synthesis.
• This method enhances the ability to quantify and understand DIF magnitudes in practical settings.