The Rapid Online Cognitive Assessment for the Detection of Neurocognitive Disorder: Open-Label Study
Calvin Howard1,2,3, Amy Johnson2, Joseph Peedicail2
1Department of Neurology, Brigham and Women's Hospital, Harvard Medical School, 60 Fenwood Road 1st Floor, Boston, MA, 02115, United States.
Background:
The rising prevalence of dementia necessitates a scalable solution to cognitive screening. Paper-based cognitive screening examinations are well-validated but minimally scalable. If a digital cognitive screening examination could replicate paper-based screening, it may improve scalability while potentially maintaining the performance of these well-validated paper-based tests. Here, we evaluate the Rapid Online Cognitive Assessment (RoCA), a remote and self-administered digital cognitive screening examination.
Objective:
The objective of this study was to validate the ability of RoCA to reliably evaluate patient input, identify patients with cognitive impairment relative to the established tests, and evaluate its potential as a screening tool.
Methods:
RoCA uses a convolutional neural network to evaluate a patient's ability to perform common cognitive screening tasks: wireframe diagram copying and clock drawing tests. To evaluate RoCA, we compared its evaluations with those of established paper-based tests. This open-label study consists of 46 patients (age range 33-82 years) who were enrolled from neurology clinics. Patients completed the RoCA screening examination and either Addenbrooke's Cognitive Examination-3 (ACE-3, n=35) or Montreal Cognitive Assessment (MoCA, n=11). We evaluated 3 primary metrics of RoCA's performance: (1) ability to correctly evaluate patient inputs, (2) ability to identify patients with cognitive impairment compared to ACE-3 and MoCA, and (3) performance as a screening tool.
Results:
RoCA classifies patients similarly to gold standard paper-based tests, with a receiver operating characteristic area under the curve of 0.81 (95% CI 0.67-0.91; P<.001). RoCA achieved sensitivity of 0.94 (95% CI 0.80-1.0; P<.001). This was robust to multiple control analyses. Approximately 83% (16/19) of the patient respondents reported RoCA as highly intuitive, with 95% (18/19) perceiving it as adding value to their care.
Conclusions:
RoCA may act as a simple and highly scalable digital cognitive screening examination. However, due to the limitations of this study, further work is required to evaluate the ability of RoCA to be generalizable across patient populations, assess its performance in an entirely remote manner, and analyze the effect of digital literacy.
