Estimating Trends With Differential Item Functioning: A Comparison of Five IRT-Based Approaches

Area of Science:

• Psychometrics
• Educational Measurement
• Statistical Modeling

Background:

• Longitudinal assessments rely on tests to estimate trends over time.
• Item parameter drift (IPD) can distort time-point comparisons, requiring robust statistical methods.
• Accurate trend estimation is crucial for valid longitudinal data interpretation.

Purpose of the Study:

• To compare five trend-estimation approaches under item parameter drift (IPD) using the two-parameter logistic (2PL) model.
• To evaluate the performance of concurrent calibration, fixed calibration, robust linking, partial invariance, and regularized estimation.
• To identify the most effective method for accurate trend estimation in longitudinal assessments with IPD.

Main Methods:

• Compared five trend-estimation approaches: concurrent calibration, fixed calibration, robust linking (Haberman, Haebara with Lp or L0 losses), partial invariance (using likelihood-ratio tests or RMSD), and regularized estimation (SBIC).
• Evaluated bias and relative root mean square error (RMSE) for trend estimates (mean and SD) at T2.
• Utilized synthetic longitudinal reading data for an empirical example.

Main Results:

• Regularized estimation with SBIC demonstrated the best performance, exhibiting low bias and RMSE across conditions.
• Robust linking methods, particularly Haberman linking with L0 loss, showed strong performance, outperforming partial invariance approaches under unbalanced IPD.
• Concurrent and fixed calibration yielded the poorest trend recovery under unbalanced IPD.

Conclusions:

• Regularized estimation with SBIC is recommended for accurate longitudinal trend estimation when item parameters may drift.
• Robust linking methods offer a viable alternative, especially Haberman linking with L0 loss, for handling IPD.
• Concurrent and fixed calibration methods are not suitable for longitudinal studies with significant item parameter drift.