PhraseAug: An Augmented Medical Report Generation Model With Phrasebook

Area of Science:

• Artificial Intelligence
• Medical Informatics
• Natural Language Processing

Background:

• Medical report generation automates accurate diagnostic reports from medical images, aiding radiologists and improving disease diagnosis efficiency.
• Fine-grained alignment between medical images and reports is crucial for cross-modal generation but challenged by visual-linguistic biases from radiologists' writing styles.

Purpose of the Study:

• To develop an augmented two-stage medical report generation model (PhraseAug) that addresses visual-linguistic bias.
• To introduce a phrasebook as an intermediate modality for discretizing reports and facilitating fine-grained image-text alignment.

Main Methods:

• A phrasebook, comprising key noun phrases including disease terms and synonymous phrases, was introduced to represent discretized medical reports.
• An augmented two-stage model (PhraseAug) was developed, integrating medical images, clinical histories, and writing styles.
• The first stage extracts features and predicts key phrases using the phrasebook; the second stage generates reports based on these phrases.

Main Results:

• The PhraseAug model demonstrated improved performance in generating diverse medical reports by adapting to different writing styles.
• Experiments on the IU-Xray and MIMIC-CXR datasets showed PhraseAug outperforming state-of-the-art baselines.
• The phrasebook effectively identified synonymous sentences and promoted fine-grained alignment between medical images and reports.

Conclusions:

• The proposed PhraseAug model effectively generates accurate and fluent medical reports by leveraging a phrasebook to mitigate visual-linguistic bias.
• The phrasebook approach facilitates better cross-modal alignment and enables the generation of diverse reports adaptable to various writing styles.
• PhraseAug represents a significant advancement in automated medical report generation, enhancing efficiency and accuracy in clinical practice.