Test bias in a cognitive test: differential item functioning in the CASI.

Area of Science:

• Psychometrics
• Cognitive Assessment
• Gerontology

Background:

• Test bias, specifically differential item functioning (DIF), is crucial for establishing construct validity.
• DIF occurs when item success probabilities differ between groups, controlling for overall ability.
• The Cognitive Assessment Screening Instrument (CASI) is widely used for assessing cognitive function in elderly populations.

Purpose of the Study:

• To analyze differential item functioning (DIF) within the Cognitive Assessment Screening Instrument (CASI).
• To evaluate a novel ordinal logistic regression technique for DIF assessment.
• To compare DIF prevalence between traditional CASI scoring and item response theory (IRT) scoring.

Main Methods:

• Utilized data from a large cohort study of elderly adults.
• Developed and applied an ordinal logistic regression model to detect DIF.
• Examined DIF across demographic variables: ethnicity, gender, education, and age.

Main Results:

• A significant number of CASI items exhibited DIF concerning demographic variables.
• Traditional CASI scoring identified more DIF items than IRT scoring.
• IRT scoring demonstrated a potential to mitigate the impact of DIF.

Conclusions:

• The prevalence of DIF in CASI items suggests potential bias, questioning previous findings of group differences in cognitive function.
• The developed DIF detection technique is effective for psychometric test evaluation.
• IRT scoring offers advantages in reducing test bias compared to traditional scoring methods.