LUMEN: LONGITUDINAL MULTI-MODAL RADIOLOGY MODEL FOR PROGNOSIS AND DIAGNOSIS

Area of Science:

• Artificial Intelligence in Medicine
• Medical Imaging Analysis
• Radiology Decision Support

Background:

• Large vision-language models (VLMs) show promise for clinical applications, particularly in radiology decision support.
• Analyzing longitudinal imaging data (e.g., chest X-rays - CXR) is crucial for accurate diagnosis and prognosis but is time-consuming.
• Existing VQA interfaces lack specialized optimization for longitudinal CXR interpretation.

Purpose of the Study:

• To introduce LUMEN, a novel training framework for longitudinal CXR interpretation.
• To enhance prognostic and diagnostic performance using multi-image and multi-task instruction fine-tuning.
• To develop and evaluate a prognostic VQA task using a novel longitudinal instruction-following dataset.

Main Methods:

• Developed LUMEN, a training framework optimized for longitudinal CXR interpretation.
• Employed multi-image and multi-task instruction fine-tuning.
• Created a new instruction-following dataset for prognostic VQA tasks, using MIMIC-CXR and Medical-Diff-VQA datasets.

Main Results:

• LUMEN demonstrated significant improvements over baseline models in diagnostic VQA tasks.
• The framework showed promising potential for prognostic capabilities in CXR analysis.
• The novel longitudinal dataset enabled the development of a prognostic VQA task.

Conclusions:

• Well-designed, instruction-tuned VLMs offer significant value in radiological interpretation.
• LUMEN enhances the accuracy and clinical meaningfulness of interpreting longitudinal radiological imaging.
• This work advances AI-driven decision support in clinical radiology, particularly for temporal analysis.