Deep learning from multiple experts improves identification of amyloid neuropathologies.

Area of Science:

• Neuropathology
• Artificial Intelligence
• Medical Diagnostics

Background:

• Pathologist variability in labeling pathologies hinders consistent diagnostic assessments.
• The absence of a single ground truth complicates reliable neuropathology evaluations.

Purpose of the Study:

• To develop a deep learning (DL) approach that integrates multiple expert annotations for consistent neuropathology diagnosis.
• To create a DL model robust to noisy labels and variable expert opinions in amyloid beta neuropathology.
• To improve diagnostic accuracy by leveraging a consensus of expert contributions.

Main Methods:

• Collected 100,495 annotations on 20,099 amyloid beta neuropathologies from five experts across three institutions.
• Developed DL methods trained on a consensus-of-two strategy, weighing expert contributions.
• Evaluated DL model performance against individualized annotations and expert benchmarks.

Main Results:

• DL models trained on consensus-of-two strategy showed 12.6-26% improvement in AUPRC compared to individualized annotation models.
• The consensus DL strategy outperformed individual-expert models, even on biased benchmarks.
• Prospective tests demonstrated DL models achieved human-like performance in labeling pathologies (AUPRC=0.74 cored, 0.69 CAA).

Conclusions:

• A DL approach combining multiple ground truths can yield consistent diagnoses informed by variable expert opinions.
• This method provides a robust means to establish a common-ground DL model for neuropathology.
• The developed DL strategy addresses inter-observer variability and enhances diagnostic reliability in amyloid beta neuropathology.