Related Experiment Video
Updated: Aug 6, 2026
08:29
Symmetric Bihemispheric Postmortem Brain Cutting to Study Healthy and Pathological Brain Conditions in Humans
Published on: December 18, 2016
Assessing AI and Neurologist Diagnostic Reasoning Against Neuropathological Ground Truth
Yu Leng1, Ayush Noori1,2,3, John R Dickson1,2
1Department of Neurology, Massachusetts General Hospital, Boston, MA, 02114, USA.
Medrxiv : the Preprint Server for Health Sciences
|July 17, 2026
Summary
A new benchmark dataset, NeuroBench, and a diagnostic AI system, DIAGNO, were developed for neurological disorders. DIAGNO demonstrated expert-level accuracy, matching or exceeding neurologists in differential diagnosis based on neuropathological ground truth.
Area of Science:
- Neurology
- Artificial Intelligence
- Medical Diagnostics
Background:
- Accurate differential diagnosis of complex neurological disorders is challenging due to overlapping symptoms.
- Current AI evaluation often relies on clinician consensus, not biological ground truth.
- A neuropathologically confirmed benchmark for AI in neurology is needed.
Purpose of the Study:
- To introduce NeuroBench, a benchmark dataset with neuropathologically confirmed diagnoses for evaluating AI in neurology.
- To develop and evaluate DIAGNO, a large language model-based system for neurological diagnosis.
Main Methods:
- Created NeuroBench with 203 retrospective cases and autopsy-confirmed diagnoses.
- Developed DIAGNO, a confidence-aware LLM using retrieval-augmented generation.
- Assessed DIAGNO and neurologists against neuropathological ground truth using blinded adjudicators.
Main Results:
- NeuroBench included 79 unique neuropathological diagnoses.
- DIAGNO matched or outperformed neurologists in top-3 accuracy (0.67 vs. 0.63) and taxonomy-level accuracy (0.74 vs. 0.66).
- DIAGNO showed favorable clinician ratings for clinical utility and safety in real-world evaluations.
Conclusions:
- NeuroBench sets neuropathological confirmation as the standard for evaluating diagnostic AI in neurology.
- DIAGNO achieved expert-level diagnostic performance, outperforming other AI models.
- DIAGNO shows potential as a clinical decision-support tool in neurology.
