Diagnostic Captioning by Cooperative Task Interactions and Sample-Graph Consistency.

Area of Science:

• Medical Imaging and Artificial Intelligence
• Natural Language Processing in Healthcare

Background:

• Radiographic images present challenges for diagnostic captioning due to subtle visual differences.
• Accurate narration of clinical importance in medical reports requires robust image-text feature correlation.

Purpose of the Study:

• To develop a novel self-boosting framework for enhanced diagnostic captioning.
• To improve the learning of tightly correlated image and text features for medical report generation.

Main Methods:

• A self-boosting framework integrating two strategies: image-text matching (ITM) and report generation (RG).
• Joint training of ITM and RG branches, where generated reports refine ITM for better alignment.
• Alignment of image-sample and report-sample spaces using graph-based learning for consistent embeddings.

Main Results:

• The proposed framework demonstrates superior performance on medical report generation benchmarks.
• Effective learning of fine-grained visual differences crucial for clinical importance.
• Superiority validated on the large-scale MIMIC-CXR dataset.

Conclusions:

• The self-boosting framework significantly enhances diagnostic captioning by improving image-text feature correlation.
• The dual-strategy approach enables progressive self-improvement without external resources.
• The method shows promise for generating high-quality, clinically relevant medical reports.