Simulating doctors' thinking logic for chest X-ray report generation via Transformer-based Semantic Query learning.

Area of Science:

• Artificial Intelligence
• Medical Imaging Analysis
• Natural Language Processing

Background:

• Medical report generation requires doctors to observe, understand, and describe images from multiple perspectives.
• Existing methods may not fully capture the nuanced relationship between visual information and descriptive language in medical reports.

Purpose of the Study:

• To propose an innovative Transformer-based Semantic Query learning paradigm (TranSQ) for automated medical report generation.
• To improve the effectiveness and clinical utility of generated medical reports.

Main Methods:

• TranSQ learns an intention embedding set and performs semantic queries on visual features.
• It generates intent-compliant sentence candidates and forms coherent reports.
• A bipartite matching mechanism is used during training to align intention embeddings with sentences, incorporating medical concepts.

Main Results:

• The TranSQ model demonstrated superior performance compared to state-of-the-art models on the IU X-ray and MIMIC-CXR radiology reporting datasets.
• Experimental results confirmed the model's effectiveness in generation and clinical efficacy.
• Ablation studies validated the innovation and interpretability of the TranSQ approach.

Conclusions:

• The proposed TranSQ paradigm offers a significant advancement in automated medical report generation.
• This approach enhances the accuracy and clinical relevance of radiology reports.
• The method provides a novel way to integrate semantic understanding with visual data for medical text generation.