Initial specialist validation of clinical decision support recommendations from a machine learning-enabled digital...

Disease-modifying therapies for Alzheimer's disease (AD) heighten demand for scalable tools enabling primary care providers (PCPs) to detect cognitive impairment and triage patients for appropriate evaluation. The brief tablet-based Linus Health Core Cognitive Evaluation (CCE) integrates the Digital Clock and Recall (DCR) and Life and Health Questionnaire (LHQ) to generate clinical decision support (CDS) and decision-tree pathways.

We conducted a retrospective specialist content-validity study (June 15-27, 2023) using a modified RAND/UCLA Appropriateness Method. Five board-certified cognitive/behavioral neurologists independently rated CDS recommendations and nine predefined pathway parts for patients aged ≥55 across 21 de-identified reports. Items scored 1-9 were summarized as pooled medians with interquartile ranges; medians ≥7 indicated appropriateness. Agreement among experts was quantified using ICC [2,k] and ICC [2,1].

All cognitive-impairment recommendations met the threshold. All seven borderline/impaired-DCR pathways were appropriate (median 7-8). Two pathways fell below threshold: cognitively unimpaired individuals with Green DCR scores (median 6) and a preliminary anti-amyloid treatment referral pathway (median 5). Agreement was moderate per patient [median ICC(2,k) = 0.61] and lower for individual diagnostic-concern recommendations [median ICC(2,k) = 0.25], reflecting specialist heterogeneity on borderline non-cognitive items and ceiling effects on high-rated items.

Cognitive neurologists judged CCE-derived CDS appropriate for PCP workup and referral decisions in older adults with suspected cognitive impairment. Findings support initial content validity of assessment-linked CDS, identify refinement priorities in low-risk and emerging-therapy pathways, and motivate planned PCP appropriateness and prospective implementation studies.