Use of Large Language Models to Determine the Surveillance Colonoscopy Interval: A Bi-Institutional Validation Study.

Area of Science:

• Gastroenterology
• Medical Informatics
• Artificial Intelligence in Medicine

Background:

• Determining post-polypectomy colonoscopy surveillance intervals requires synthesizing complex guideline algorithms.
• Manual application of these guidelines, such as the United States Multi-Society Task Force (USMSTF) recommendations, is prone to errors.
• Automated tools, including large language models (LLMs), are needed to improve adherence to surveillance guidelines.

Purpose of the Study:

• To evaluate the performance of a large language model (LLM) in determining guideline-concordant post-polypectomy surveillance intervals.
• To assess LLM accuracy on a real-world dataset of 1000 colonoscopy and pathology report impressions.

Main Methods:

• A cohort of 1000 colonoscopy and pathology reports from 2023-2024 was analyzed.
• The GPT-4o LLM, with a custom prompt detailing the USMSTF surveillance algorithm, was used to determine surveillance intervals.
• The experiment was repeated 10 times to ensure reliability.

Main Results:

• The LLM achieved an average accuracy of 94.6% across 10 experiments.
• Accuracy was high regardless of the originating institution or presence of upper GI endoscopy data.
• Accuracy was 95.8% for cases with 1-3 polyps versus 88.2% for cases with 4+ polyps (p<0.001).

Conclusions:

• Large language models (LLMs) demonstrate high accuracy in applying post-polypectomy surveillance guidelines.
• Custom-prompted LLMs offer a reliable automated solution for determining colonoscopy surveillance intervals.
• This technology has the potential to significantly improve guideline adherence in clinical practice.