Assessing Measurement Invariance in ASWB Exams: Regulatory Research Proposal to Advance Equity

Area of Science:

• Psychometrics
• Social Work Education
• Measurement Invariance

Background:

• Social workers from minoritized groups face disparities in passing licensing exams.
• Licensing exams are crucial for professional practice.
• Understanding exam fairness is essential for equitable access to the profession.

Purpose of the Study:

• To investigate the measurement equivalence (invariance) of social work licensing examinations.
• To simulate response data and analyze the impact of differential item functioning (DIF) on overall test performance.
• To assess the relationship between item-level and test-level functioning in the context of licensing exams.

Main Methods:

• Simulated response data for 15 multiple-choice questions.
• Utilized the R mirt package to fit a 2-parameter logistic (2PL) model.
• Generated data with five items exhibiting DIF and analyzed their impact on test and item characteristic curves.

Main Results:

• Small amounts of differential item functioning aggregated into differential test functioning.
• The overall effect size of DIF on test functioning was found to be small.
• This outcome represents a potential finding from analyses of social work licensing exams like the ASWB exams.

Conclusions:

• Differential test functioning is a key component of measurement invariance studies.
• Psychometric standards mandate item-level and test-level measurement invariance assessments to mitigate bias.
• Analyzing test characteristic curves aids in identifying content-irrelevant variance, such as unfairness or unreliable pass scores.