Detecting differential item functioning in behavioral indicators across parallel forms

Area of Science:

• Psychometrics
• Educational Measurement
• Psychological Assessment

Background:

• Classical Test Theory (CTT) relies on parallel test forms, but direct verification is limited by unobservable true scores.
• Traditional methods for assessing test form parallelism face inherent limitations.
• Differential Item Functioning (DIF) offers a novel framework to address these limitations.

Purpose of the Study:

• To propose and illustrate a novel approach for verifying the parallelism of test forms using DIF.
• To shift the focus from total test scores to individual item performance in assessing parallelism.
• To introduce the "differential functioning of behavioral indicators" (DFBI) as a method to analyze item-level parallelism.

Main Methods:

• Applied several DIF techniques to analyze the performance of a single group on parallel items.
• Examined 18 items from two parallel forms of the Attention Deficit-Hyperactivity Disorder Scale.
• Utilized a dataset of 527 participants to illustrate the proposed approach.

Main Results:

• 12 out of 18 items (66.6%) exhibited statistically significant differential functioning (p < .01) based on the Mantel χ² statistic.
• Standardization revealed that DIF items were equally distributed, with half favoring Form A and half favoring Form B.
• This indicates potential non-parallel functioning at the item level despite overall test form equivalence.

Conclusions:

• The "differential functioning of behavioral indicators" (DFBI) provides valuable item-level information on test form parallelism.
• DFBI complements traditional analyses based on total scores, offering a more nuanced understanding of equivalence.
• This approach enhances the psychometric rigor in evaluating parallel test forms.