Using Large Language Models to Determine Reasons for Missed Colon Cancer Screening Follow-Up

Identifying reasons for missed preventive care, such as follow-up colonoscopy after an abnormal stool-based colon cancer screening test, is critical for quality improvement initiatives. However, manual chart review is time-consuming and costly. This study serves as a proof-of-concept demonstration of using large language models (LLMs) to automate the identification of documented reasons for care gaps.

In this cross-sectional study of patients aged 45 years or older at the University of California, San Francisco with an abnormal outpatient fecal immunochemical test/fecal occult blood test (FIT/FOBT) who did not undergo a colonoscopy within 90 days of the abnormal test, the authors investigated the potential of an LLM to determine (a) whether reasons for a lack of follow-up colonoscopy are documented in the clinical notes and (b) whether an LLM can accurately classify those reasons into clinically meaningful categories.

A total of 846 patients with abnormal FIT/FOBTs who did not receive a colonoscopy within 90 days of the abnormal test were included in this study. Based on LLM categorization of patient note content, 270 (31.9%) patients did not have any reference to colonoscopy/colorectal cancer screening in their notes, 379 (44.8%) patients had mentions of colonoscopy/colorectal cancer screening without explicit reasons for not having a colonoscopy provided, and 197 (23.3%) patients had notes detailing explicit reasons for not having a colonoscopy. Overall LLM classification accuracy was 89.0%. The most common reasons for not having a colonoscopy included refused/not interested (35.2%), comorbidities (18.7%), and patient unavailable (16.8%).

This pilot study suggests that LLMs can accurately categorize reasons for the absence of follow-up colonoscopy after an abnormal FIT/FOBT. These results suggest that LLMs have the potential to automate chart review for quality improvement initiatives.