Binary classification threatens the validity of cognitive impairment detection

Area of Science:

• Neuropsychology
• Oncology
• Cognitive Assessment

Background:

• Cognitive impairment prevalence varies in patient populations despite standardized testing.
• The International Cognition and Cancer Task Force (ICCTF) proposed standard cutoffs for cognitive impairment in cancer patients.
• Harmonizing operationalization of cognitive impairment is crucial for accurate assessment.

Purpose of the Study:

• To evaluate how binary classification of cognitive impairment affects agreement between two comparable test batteries.
• To assess the impact of ICCTF criteria on cognitive impairment classification in cancer patients.
• To investigate the agreement between traditional and online neuropsychological test formats.

Main Methods:

• Two hundred non-central nervous system (non-CNS) cancer patients completed traditional and online test batteries.
• Cognitive impairment was defined using ICCTF criteria (≥ 1.5 or ≥ 2 standard deviations below normative means).
• Agreement was assessed using Cohen's kappa (κ), with additional Monte Carlo simulations.

Main Results:

• Total scores between traditional and online tests showed a high correlation (.78).
• Proportions of impaired patients were similar across methods (40% traditional vs. 38% online).
• Within-person agreement in impairment classification was only fair (κ = .35), with low agreement in simulations (κ = .33–.41).

Conclusions:

• Binary classification can lead to dissimilar impairment identification between highly similar test batteries.
• Inherent low within-person agreement exists between assessment methods using binary classification.
• Modern statistical tools may enhance the validity of cognitive impairment detection.