HiVLR: Hierarchical Vision-Language Reasoning for interpretable zero-shot radiography image understanding

Area of Science:

• Artificial Intelligence
• Medical Imaging
• Natural Language Processing

Background:

• Medical vision-language pre-training shows promise for image understanding.
• Existing methods struggle with zero-shot accuracy and interpretability in medical diagnosis.

Purpose of the Study:

• To develop a framework that improves accuracy and interpretability in medical image diagnosis.
• To address usability and trust issues in safety-critical medical AI applications.

Main Methods:

• Proposed a Hierarchical Vision-Language Reasoning (HiVLR) framework mimicking clinical diagnostic workflows.
• Structured feature investigation into sequential rounds: pathology observation and diagnostic finding determination.
• Employed cross-attention, prompt tuning, and concept-level cross-modal compliance for feature analysis.

Main Results:

• HiVLR significantly outperformed competing approaches on diverse zero-shot image tasks.
• The framework demonstrated superior interpretability in model decision-making.
• Achieved accurate disease predictions with enhanced transparency.

Conclusions:

• HiVLR offers a more accurate and interpretable solution for medical image diagnosis.
• The framework's clinical reasoning approach enhances trust and usability in medical AI.
• HiVLR sets a new standard for zero-shot medical image understanding tasks.