Differential Item Functioning Analysis Without A Priori Information on Anchor Items: QQ Plots and Graphical Test.

Area of Science:

• Psychometrics
• Educational Measurement
• Statistical Modeling

Background:

• Differential item functioning (DIF) analysis is crucial for measurement validity.
• Traditional DIF methods rely on anchor items, which can lead to biased results if flawed.
• Existing methods may fail when many items exhibit DIF or patterns are unbalanced.

Purpose of the Study:

• To introduce a novel DIF detection method, RCD-DIF, robust to flawed anchor items.
• To provide a method that functions effectively even with unbalanced DIF patterns across items.
• To enhance the accuracy and reliability of DIF analysis in psychometric assessments.

Main Methods:

• Proposed RCD-DIF method based on the relative change of difficulty difference (RCD), independent of group mean ability.
• Utilized D-QQ plots for internal reference point identification.
• Employed RCD-QQ plots and a RCD graphical test for visual DIF examination and synchronized item/test-level analysis.

Main Results:

• The RCD-DIF method effectively identifies DIF without relying on assumed DIF-free anchor items.
• Simulation studies demonstrated the RCD graphical test maintains comparable Type I error rates to existing methods.
• The RCD graphical test exhibited greater statistical power compared to traditional approaches.

Conclusions:

• The RCD-DIF method offers a reliable alternative for DIF analysis, particularly when anchor items are suspect or DIF is widespread.
• The RCD procedure provides visual insights into DIF patterns and item-level effects.
• This approach enhances the validity of measurement scales by accurately detecting DIF.