A Feasible Machine Learning Approach to Improve Cognitive Screening in Multiple Sclerosis

Background: Cognitive impairment is a frequent manifestation of multiple sclerosis (MS). Although the Brief International Cognitive Assessment for Multiple Sclerosis (BICAMS) is the recommended screening battery, the Montreal Cognitive Assessment (MoCA) is more commonly used in routine clinical practice. The ability of published MoCA cutoffs to detect MS-specific cognitive impairment remains unclear. Objective: This study aimed to evaluate the diagnostic performance of published Italian MoCA cutoffs for MS-specific cognitive impairment and develop a machine learning (ML) pipeline to improve screening accuracy. Methods: We prospectively enrolled 222 people with MS who underwent both MoCA and BICAMS assessment. Cognitive impairment was defined as performance below the 5th percentile on at least one BICAMS test. The sensitivity, specificity, and balanced accuracy of four published Italian MoCA cutoffs were calculated. An elastic-net penalized logistic regression model incorporating age, education, and raw MoCA score was trained using nested five-fold cross-validation to predict cognitive impairment. We additionally evaluated the impact of disability and fatigue on cognitive impairment and model performance. Results: Cognitive impairment was identified in 62 (27.9%) participants. Raw MoCA scores were significantly lower in cognitively impaired participants than in cognitively preserved participants (22.68 ± 4.29 vs. 25.77 ± 3.67; p < 0.0001). Published MoCA cutoffs, however, showed extreme specificity (0.92-0.99) but below-chance sensitivity (0.10-0.44), yielding low balanced accuracy (0.54-0.68). The ML model achieved good discrimination (ROC-AUC = 0.787, 95% CI 0.717-0.852), with sensitivity = 0.69, specificity = 0.72, and balanced accuracy = 0.71. Conclusions: Published Italian MoCA cutoffs have insufficient sensitivity for screening MS-specific cognitive impairment. An ML-based scoring approach substantially improves sensitivity while maintaining acceptable specificity, offering a practical tool to enhance cognitive screening in MS.