Justifying diagnosis decisions by deep neural networks

Area of Science:

• Artificial Intelligence in Medicine
• Medical Imaging Analysis
• Deep Learning Applications

Background:

• Growing interest in applying deep learning to medical applications.
• Need for explainable AI (Artificial Intelligence) in clinical workflows for clinician validation.
• Current methods for AI-driven medical diagnosis lack sufficient justification mechanisms.

Purpose of the Study:

• To develop an integrated approach using visual and textual data for medical diagnosis determination and justification.
• To enable machine learning-aided medical workflows by providing interpretable outcomes for clinicians.
• To enhance the validity assessment of AI-generated diagnoses.

Main Methods:

• Utilized deep learning to map frontal X-ray images to continuous textual representations.
• Decoded textual representations into diagnoses and associated textual justifications.
• Generated realistic X-ray images for nearest alternative diagnoses to provide further explanatory data.
• Employed multi-task training with multiple loss functions.

Main Results:

• Demonstrated significant outperformance of the justification mechanism compared to saliency map methods in a clinical expert opinion study.
• Achieved excellent diagnosis accuracy and captioning quality.
• Validated the approach on X-ray data from the Indiana University hospital network.

Conclusions:

• The proposed integrated approach effectively determines and justifies medical diagnoses from X-ray images.
• The method significantly enhances the interpretability and validity of AI-generated medical diagnoses for clinicians.
• This work advances the application of explainable AI in medical imaging, improving diagnostic workflows.