GastroGPT Pioneering Specialized AI in Gastroenterology: Strengths, Pitfalls, and the Road to Clinical Integration

GastroGPT, a transformer-based large language model, has been developed specifically for gastroenterology. It exhibited improved ability in clinical tasks compared to a general-purpose model such as GPT-4, Bard and Claude. GastroGPT was developed by Cem Simsek, MD, and was presented at UEG Week 2023. The GastroGPT dataset is adapted on 1.2 million tokens, including peer-reviewed content from leading gastroenterology journals, clinical guidelines and 10 000 synthetic GI vignettes. In 10 simulated cases of inflammatory bowel disease, cases of endoscopy, and hepatology, GastroGPT achieved a mean score of 8.1 ± 1.8 on a 10-point Likert scale. GastroGPT achieved higher mean scores compared to comparators (p < 0.001) on six out of seven tasks and included tasks such as patient history acquisition, recommendation for referral, and patient education. Its reproducibility and consistency across task complexities indicate its potential in situations of resource limitation. While it remains limited by its reliance on simulated cases, some participant selection and exposure bias attributed to training data, and lack of appropriate comparisons with medical-specific models such as OpenEvidence, there remains the need for future real-world trials and multimodal integrations within workflows to evaluate GastroGPT's transformation potential in improving gastroenterology workflows and patient care.