Joint Imbalance Adaptation for Radiology Report Generation

Area of Science:

• Artificial Intelligence
• Medical Imaging
• Natural Language Processing

Background:

• Radiology report generation translates medical images into clinical descriptions.
• Data imbalance, including infrequent medical tokens and a prevalence of normal findings, challenges report generation models.
• Existing studies rarely address these specific imbalance issues.

Purpose of the Study:

• To address data imbalance challenges in radiology report generation.
• To improve the robustness and accuracy of automated radiology reports.
• To mitigate the underperformance caused by infrequent tokens and abnormal labels.

Main Methods:

• Proposed a Joint Imbalance Adaptation (JIMA) model.
• Leveraged token and label imbalance for improved task robustness.
• Employed a hard-to-easy learning strategy to focus on rare labels and clinical tokens.
• Predicted entity distributions from images to generate reports.

Main Results:

• Achieved notable improvements (16.75% - 50.50% on average) on IU X-ray and MIMIC-CXR datasets.
• Demonstrated enhanced handling of infrequent tokens and abnormal labels.
• Human evaluation and case studies confirmed the generation of more clinically accurate reports.

Conclusions:

• Data imbalance significantly impacts radiology report generation performance.
• The proposed curriculum learning strategy effectively reduces data imbalance impacts.
• The JIMA approach offers a promising direction for improving automated medical report generation.