Automated RECIST tumor response classification through prompt-guided large language models

Area of Science:

• Medical Imaging and Radiology
• Artificial Intelligence in Healthcare
• Oncology

Background:

• Accurate tumor response assessment is crucial for cancer treatment evaluation.
• Manual classification of radiology reports can be time-consuming and prone to variability.
• Large language models (LLMs) show potential for automating clinical text analysis.

Purpose of the Study:

• To evaluate an offline, general-purpose LLM's ability to classify radiology reports according to Response Evaluation Criteria in Solid Tumors (RECIST) guidelines.
• To assess the impact of different prompting strategies (zero-shot, few-shot, chain-of-thought) on classification accuracy.
• To ensure privacy-preserving tumor response assessment without model fine-tuning.

Main Methods:

• An in-house, offline LLaMA-3.3 (70B) model was used to process CT imaging reports from oncology patients.
• Reports were classified into RECIST categories (Baseline, Complete Response, Partial Response, Stable Disease, Progressive Disease) using three prompting strategies.
• Model performance was benchmarked against expert labels using accuracy, precision, recall, and F1 scores.

Main Results:

• The LLM achieved strong classification performance across all prompting strategies.
• Chain-of-thought prompting yielded the best results, with a micro F1 score of 0.81.
• Model predictions showed good alignment with human expert assessments.
• The offline system maintained strict data privacy compliance.

Conclusions:

• Prompt-driven LLMs can accurately and reliably classify tumor response categories from real-world radiology reports.
• Offline LLM deployment, coupled with optimized prompting, offers a scalable and privacy-preserving solution for oncology report interpretation.
• This approach has the potential to enhance consistency and efficiency in clinical decision support.