Related Experiment Videos
Modern psychometric methods for detection of differential item functioning: application to cognitive assessment
J A Teresi1, M Kleinman, K Ocepek-Welikson
1Columbia University, Stroud Center, New York, NY 10032, USA. Teresimeas@aol.com
Statistics in Medicine
|June 9, 2000
Summary
This study found minimal bias in the Mattis Dementia Rating Scale Attention subscale across education levels, though some items showed minor differential item functioning (DIF). These psychometric methods are crucial for unbiased dementia epidemiology and clinical trials.
Area of Science:
- Psychometrics
- Neuropsychological Assessment
- Dementia Epidemiology
Background:
- Cognitive screening tests can exhibit performance disparities across demographic groups (education, ethnicity, race).
- Such bias has significant implications for dementia epidemiology research, yet is understudied.
- Differential item functioning (DIF) is a key indicator of potential bias in psychometric measures.
Purpose of the Study:
- To examine the performance of the Attention subscale of the Mattis Dementia Rating Scale (MDRS).
- To assess for differential item functioning (DIF) across different education groups using modern psychometric methods.
- To evaluate the utility of item response theory (IRT) models for detecting bias in neuropsychological subtests.
Main Methods:
- Applied item response theory (IRT) models, including two- and three-parameter logistic models and a polytomous model, to the MDRS Attention subscale data.
- Utilized confirmatory factor analysis and item-fit statistics (BIMAIN) to identify problematic items.
- Employed model-based tests of DIF (MULTILOG) and cross-validation to assess performance differences across education subgroups.
Main Results:
- The MDRS Attention subscale demonstrated high reliability (KR-20 = 0.92) and IRT-based reliability estimates ranging from 0.65 to 0.97.
- Most items showed minimal DIF, with nearly identical item characteristic curves across education groups.
- Four items exhibited problematic performance, including one (digit span backwards) with a low discrimination parameter and significant DIF, particularly non-uniform DIF observed in one item.
Conclusions:
- The MDRS Attention subscale appears largely free of significant bias related to education.
- The applied psychometric methods (IRT, DIF analysis) are effective for detecting bias in neuropsychological measures.
- These methods are valuable for ensuring culture-fair classifications in dementia screening and for analyzing site differences in multi-site clinical trials.