Fine-Tuned Large Language Model for Extracting Pretreatment Pancreatic Cancer According to Computed Tomography...

Area of Science:

• Artificial Intelligence in Radiology
• Natural Language Processing for Medical Text Analysis
• Machine Learning for Cancer Detection

Background:

• Accurate identification of pretreatment pancreatic cancer from radiology reports is crucial for timely patient management.
• Manual review of computed tomography (CT) radiology reports is time-consuming and prone to variability.
• Large language models (LLMs) show potential in automating medical text analysis.

Purpose of the Study:

• To evaluate the performance of a fine-tuned LLM in extracting pretreatment pancreatic cancer information from CT radiology reports.
• To compare the LLM's extraction accuracy and efficiency against human readers.

Main Methods:

• A retrospective study utilized 2690, 886, and 378 CT reports for training, validation, and testing, respectively.
• A pre-trained Bidirectional Encoder Representation from the Transformers Japanese model was fine-tuned on the CT report datasets.
• The fine-tuned LLM and three human readers classified reports into 'no pancreatic cancer,' 'post-treatment,' or 'pretreatment cancer present' categories.

Main Results:

• The fine-tuned LLM achieved an overall accuracy of 0.942, comparable to human readers (0.984, 0.979, 0.947).
• The LLM demonstrated high sensitivity in differentiating all three pancreatic cancer groups (0.944/0.960/0.921).
• The LLM required only 49 seconds for classification, significantly faster than human readers (2689-4887 seconds).

Conclusions:

• Fine-tuned LLMs can effectively extract pretreatment pancreatic cancer data from CT radiology reports.
• The LLM offers a highly accurate and significantly more efficient alternative to manual review by human readers.
• This technology has the potential to streamline radiological workflow and improve diagnostic efficiency.