Policy-Driven, Multimodal Deep Learning for Predicting Visual Fields from the Optic Disc and OCT Imaging.

Area of Science:

• Ophthalmology
• Medical Imaging
• Artificial Intelligence

Background:

• Glaucoma diagnosis relies on correlating visual field (VF) sensitivity with optic disc structure.
• Current methods for structure-function mapping can be limited.
• Deep learning (DL) offers potential for advanced image analysis and prediction.

Purpose of the Study:

• To develop and validate a deep learning system for predicting visual field sensitivity at each point.
• To derive a structure-function map using optical coherence tomography (OCT) and optic disc imaging.
• To fuse predictions from single-modality DL models for improved accuracy.

Main Methods:

• Retrospective study of 6437 patients with glaucoma.
• Paired OCT and infrared reflectance (IR) optic disc images with VF data within 7 days.
• Trained EfficientNet B2 DL models for single-modality prediction and a policy DL model for fusion.

Main Results:

• A multimodal, policy DL model significantly improved prediction accuracy (PMAe) compared to single modalities.
• The fusion model showed statistically significant improvements (P < 0.0001).
• Occlusion masking confirmed data-driven, feature-agnostic learning of structure-function relationships.

Conclusions:

• The multimodal policy DL model demonstrated superior performance in predicting visual fields.
• The system provides explainable confidence maps for data fusion.
• This approach offers a novel pathway for investigating the structure-function relationship in glaucoma.