A Novel Method for Detecting Intersectional DIF: Multilevel Random Item Effects Model with Regularized Gaussian...

Area of Science:

• Psychometrics
• Educational Measurement
• Sociology

Background:

• Ensuring assessment fairness is crucial, with differential item functioning (DIF) screening being a key method.
• Traditional DIF methods often overlook the complex interactions among multiple demographic identities, focusing only on main effects.
• The intersectionality framework provides a lens to understand how combined demographic factors can uniquely impact individuals.

Purpose of the Study:

• To propose a novel item response theory (IRT) approach for detecting intersectional DIF, which accounts for interactions among demographic variables.
• To introduce the concept of intersectional impact, examining interaction effects on group-level mean ability.
• To develop four distinct models for detecting various forms of intersectional DIF, including uniform and non-uniform DIF, with and without intersectional impact.

Main Methods:

• Utilizing an intersectionality framework within an item response theory (IRT) model.
• Implementing fixed effects to control for traditional DIF and random item effects to capture intersectional DIF.
• Developing a regularized Gaussian variational expectation-maximization algorithm for efficient model estimation.

Main Results:

• The proposed methods effectively detect intersectional uniform DIF (UDIF).
• Detection of intersectional non-uniform DIF (NUDIF) was found to be more limited compared to UDIF.
• Simulation studies validated the utility of the developed models for identifying intersectional DIF.

Conclusions:

• The novel IRT approach effectively addresses the limitations of traditional DIF methods by incorporating intersectionality.
• The developed models offer a more nuanced understanding of assessment fairness by considering the interplay of multiple demographic factors.
• Further research may be needed to enhance the detection capabilities for intersectional non-uniform DIF.