Aligning, Autoencoding and Prompting Large Language Models for Novel Disease Reporting

Area of Science:

• Artificial Intelligence in Medical Imaging
• Natural Language Processing for Healthcare
• Deep Learning for Radiology

Background:

• Automatic radiology report generation is crucial for diagnostic radiology.
• Current methods require large, annotated datasets, which are unavailable for novel diseases.
• This limitation hinders timely diagnosis and reporting of emerging health threats.

Purpose of the Study:

• To develop a prompt-based deep learning framework (PromptLLM) for accurate and efficient radiology report generation for novel diseases.
• To overcome the data scarcity issue in training models for rare or newly identified diseases.
• To enable rapid knowledge acquisition for reporting novel diseases with limited labeled data.

Main Methods:

• PromptLLM aligns visual radiology images with textual reports to learn cross-modal knowledge.
• It autoencodes large language models (LLMs) using unlabeled data from novel diseases to capture specific knowledge and writing styles.
• The framework then prompts the LLM with learned information to generate reports for novel diseases.

Main Results:

• PromptLLM demonstrates accurate novel disease reporting with significantly limited labeled data (1% of training data).
• Experiments on COVID-19 and diverse thorax diseases show competitive performance compared to existing methods.
• The approach effectively reduces the dependency on large, annotated medical datasets.

Conclusions:

• PromptLLM offers an efficient solution for generating radiology reports for novel diseases, even with minimal labeled data.
• This framework can significantly impact early-stage analysis and reporting during novel disease outbreaks.
• It relaxes the reliance on extensive labeled datasets, making AI-assisted radiology more adaptable to emerging health challenges.