Inferring cancer disease response from radiology reports using large language models with data augmentation and...

Area of Science:

• Artificial Intelligence in Oncology
• Natural Language Processing for Medical Reports
• Machine Learning in Radiology

Background:

• Accurate assessment of cancer disease response is crucial for patient management and research.
• Manual review of free-text radiology reports is time-consuming and prone to variability.
• Large language models (LLMs) offer potential for automated analysis of clinical text.

Purpose of the Study:

• To evaluate the efficacy of LLMs in inferring cancer disease response from radiology reports.
• To compare different machine learning models, including transformers, LSTMs, and CNNs.
• To investigate the impact of data augmentation and prompt-based fine-tuning on model performance.

Main Methods:

• Utilized 10,602 computed tomography reports from cancer patients, classified into four response categories.
• Applied transformer models, LSTM, CNN, and conventional machine learning techniques.
• Employed data augmentation (sentence permutation) and prompt-based fine-tuning for optimization.

Main Results:

• The GatorTron transformer model achieved high accuracy (0.8916 test set, 0.8919 RECIST validation set).
• Data augmentation further enhanced accuracy to 0.8976.
• Prompt-based fine-tuning reduced training data requirements to 500 reports without significant performance loss.

Conclusions:

• LLMs show significant potential for large-scale, accurate inference of cancer disease response from radiology reports.
• Data augmentation is an effective strategy for improving the performance of these models.
• Prompt-based fine-tuning offers a method to achieve good performance with substantially reduced training datasets.