Automated report-based tracking of pancreatic cysts: implications for guideline-defined growth classification

Area of Science:

• Radiology and Medical Imaging
• Artificial Intelligence in Healthcare
• Oncology and Gastroenterology

Background:

• Pancreatic cystic lesions (PCLs) require careful monitoring for growth, a key indicator of malignancy.
• Current methods for tracking PCLs across serial radiology reports can be labor-intensive and prone to inconsistency.
• Automated methods are needed to improve the efficiency and accuracy of PCL surveillance.

Purpose of the Study:

• To develop and validate a hybrid large language model (LLM)- and rule-based algorithm for automated tracking of PCLs.
• To characterize PCL growth using both first-to-last and consecutive comparison methods across serial reports.
• To identify predictors of PCL growth meeting worrisome criteria.

Main Methods:

• A retrospective study utilized abdominal imaging reports from 2005-2024.
• A hybrid algorithm combined LLM-extracted PCL features with rule-based logic for lesion tracking.
• Algorithm performance was validated against radiologist annotations (96% agreement, Cohen's κ=0.81).

Main Results:

• The algorithm demonstrated high accuracy in tracking PCLs across 28,489 reports in 8,176 patients.
• Growth classification differed significantly between ACR and Kyoto criteria, and between first-to-last vs. consecutive comparisons.
• Larger baseline cyst size was the strongest predictor (HR 4.0) of meeting the Kyoto growth rate criterion.

Conclusions:

• A hybrid LLM-rule-based algorithm provides reliable automated longitudinal tracking of PCLs.
• PCL growth classification is dependent on the chosen guideline and comparison interval.
• Baseline cyst size is a critical factor in predicting concerning PCL growth.