Comparison of Deep Learning Tools for Optic Nerve Axon Quantification Finds Limited Generalizability on Independent...

Area of Science:

• Ophthalmology
• Neuroscience
• Biomedical Engineering

Background:

• Automated quantification of optic nerve histology using machine learning (ML) offers objective assessment of axonal injury in experimental glaucoma.
• Generalizability of these ML models to independent datasets is a critical, yet unclear, concern.

Purpose of the Study:

• To perform independent validation testing of publicly available ML models for optic nerve histology.
• To assess the generalizability of these models on a novel rat optic nerve dataset.

Main Methods:

• A scoping review guided the selection of ML models for validation.
• Three models (AxoNet, AxoNet 2.0, AxonDeepSeg) were independently validated on a dataset of 57 rat optic nerve images with 9,514 manually annotated axons.

Main Results:

• Published correlations for ML models ranged from 0.959 to 0.99, but independent validation showed reduced performance (r=0.79-0.89).
• Segmentation quality metrics indicated high precision but low recall, with Dice coefficients (0.29-0.40) significantly below benchmarks (0.81).

Conclusions:

• Deep learning models for optic nerve histology exhibit a generalizability gap, with performance decrements on independent datasets.
• Standardized validation datasets and multi-center testing are essential before widespread adoption of these tools for glaucoma research.