ChatGPT4 Outperforms Endoscopists for Determination of Postcolonoscopy Rescreening and Surveillance Recommendations

Area of Science:

• Artificial Intelligence in Medicine
• Gastroenterology
• Oncology

Background:

• The clinical impact of large language models like ChatGPT4 on gastroenterology practice remains unclear.
• Assessing ChatGPT4's accuracy in colonoscopy surveillance recommendations is crucial for its potential integration into clinical workflows.

Purpose of the Study:

• To compare the accuracy, concordance, and reliability of ChatGPT4's colonoscopy surveillance recommendations against established guidelines and current gastroenterology practice.
• To evaluate the potential of ChatGPT4 in determining appropriate colorectal cancer rescreening and surveillance intervals.

Main Methods:

• De-identified patient data (history, colonoscopy, pathology) from 505 cases were input into ChatGPT4.
• ChatGPT4's recommended follow-up intervals were compared to US Multisociety Task Force (USMSTF) guidelines and actual gastroenterologist recommendations.
• Statistical analyses included McNemar test and inter-rater reliability (Fleiss κ).

Main Results:

• ChatGPT4 provided a follow-up recommendation in 99.2% of queries.
• ChatGPT4 recommendations showed higher agreement with USMSTF guidelines (85.7%) compared to gastroenterology practice (75.4%) (P < .001).
• Good inter-rater reliability was observed between ChatGPT4 and USMSTF Panel (κ = 0.786).

Conclusions:

• ChatGPT4 demonstrates accuracy in defining colonoscopy screening intervals based on clinical data input.
• Large language models show promise for clinical applications in gastroenterology, but require additional training for widespread adoption.
• Real-world data suggest ChatGPT4 can support clinical decision-making for colorectal cancer surveillance.