A Comparative Study of the Bias Correction Methods for Differential Item Functioning Analysis in Logistic Regression...

Area of Science:

• Psychometrics
• Statistical modeling

Background:

• Logistic regression (LR) is commonly used for differential item functioning (DIF) analysis.
• Standard LR relies on asymptotic sampling distributions, which can be unreliable for rare events or imbalanced data.
• Maximum likelihood estimation (ML) in LR may yield biased results with imbalanced data.

Purpose of the Study:

• To compare the performance of regular maximum likelihood (ML) estimation with bias-corrected methods, weighted logistic regression (WLR) and Firth's penalized maximum likelihood (PML).
• To evaluate these methods for assessing DIF specifically in the context of imbalanced or rare events data.
• To investigate the impact of sample size, DIF magnitude, and data imbalance on the power and type I error rates of LR models for DIF detection.

Main Methods:

• Simulation study comparing ML, WLR, and PML estimation methods for DIF detection.
• Analysis of power and type I error rates under various conditions: sample size, DIF magnitude (0.4 and 0.8), sample size ratio, number of items, and degree of data imbalance (τ).

Main Results:

• Weighted logistic regression (WLR) demonstrated superior performance compared to ML and PML for DIF assessment with imbalanced data.
• Under severe imbalance (τ = 0.069), PML and ML showed significant reductions in power (approx. 23-30%) compared to WLR when detecting moderate to severe DIF.
• WLR consistently outperformed ML and PML in maintaining detection power for DIF in rare events data.

Conclusions:

• Weighted logistic regression (WLR) is the recommended method for assessing differential item functioning (DIF) when dealing with imbalanced or rare events data.
• Bias-corrected methods like WLR offer more reliable DIF detection than standard ML or PML in challenging data scenarios.
• The findings highlight the limitations of traditional LR-based DIF methods with skewed data distributions.