Bridging human and machine intelligence: Reverse-engineering radiologist intentions for clinical trust and adoption

Area of Science:

• Medical Imaging
• Artificial Intelligence
• Radiology

Background:

• The
• black box
• nature of AI hinders clinical adoption due to lack of transparency. Integrating AI with clinical expertise is crucial for enhancing diagnostic precision in medical imaging.

Purpose of the Study:

• To develop a novel system using Large Multimodal Models (LMM) to bridge the gap between AI predictions and radiologists' cognitive processes.
• To improve the interpretability and trustworthiness of AI in clinical practice.
• To enhance diagnostic accuracy and support medical education and training.

Main Methods:

• A novel system with two modules: Temporally Grounded Intention Detection (TGID) and Region Extraction (RE).
• TGID analyzes eye gaze heatmap videos and radiology reports to predict radiologist intentions.
• RE extracts regions of interest aligned with predicted intentions, mirroring the radiologist's focus. This is the first application of Dense Video Captioning (DVC) in medicine.

Main Results:

• The system demonstrated superior performance in generating temporally grounded intentions on the REFLACX and EGD-CXR datasets.
• Achieved strong predictive accuracy in overlap scores for medical abnormalities and effective region extraction with high Intersection over Union (IoU).
• Showcased particular effectiveness in complex cases like cardiomegaly and edema.

Conclusions:

• The proposed AI system enhances transparency and aligns with radiologists' cognitive processes, fostering trust and improving diagnostic accuracy.
• This approach offers potential for automated error correction, guiding junior radiologists, and improving training.
• Sets a precedent for human-centered, transparent, and trustworthy AI in healthcare.