Methods for the Comparison of Differential Item Functioning across Assessments.

Area of Science:

• Psychometrics
• Educational Measurement
• Psychological Assessment

Background:

• Selecting reliable and valid scales is crucial for educational and psychological evaluations.
• Ensuring fairness across different population subgroups is a key aspect of scale validity.
• Differential item functioning (DIF) and differential test functioning (DTF) are statistical methods to assess scale fairness.

Purpose of the Study:

• To explore statistical methods for comparing the magnitude of DIF/DTF between two or more assessments.
• To aid researchers in selecting the most appropriate assessment, particularly in intelligence testing.
• To provide insights into how DTF manifests across different scales.

Main Methods:

• Utilized 7 different statistical effect size statistics.
• Applied these statistics in the context of intelligence testing.
• Compared the amount and manifestation of differential test functioning (DTF) across different scales.

Main Results:

• The study demonstrated the utility of various effect size statistics for comparing DTF.
• Researchers can identify scales with minimal DTF using these methods.
• Insights were gained into the specific ways DTF manifests within different intelligence scales.

Conclusions:

• Employing a variety of effect sizes offers a robust approach to evaluating scale fairness.
• These methods assist in the selection of psychometric instruments that function equitably across diverse populations.
• Understanding the nature of DTF is as important as quantifying its magnitude when choosing assessment tools.