Chat-based Artificial Intelligence model as a source of information for Inflammatory bowel disease

Area of Science:

• Artificial Intelligence in Medicine
• Patient Education
• Gastroenterology

Background:

• Artificial intelligence (AI) models show potential for delivering scientific information to patients.
• Evaluating the accuracy of AI-generated responses is crucial for reliable patient education.

Purpose of the Study:

• To assess the accuracy and quality of answers provided by Chat GPT (Generative Pre-trained Transformer) to common questions concerning inflammatory bowel diseases (IBD).

Main Methods:

• 340 questions across various IBD domains (basic knowledge, symptoms, diagnosis, treatment, etc.) were posed to Chat GPT twice.
• Responses were evaluated for quality (poor to very good) and accuracy (comprehensive, correct but inadequate, mixed, or incorrect).

Main Results:

• Overall response quality was predominantly acceptable to good (78.2%-82.4%).
• Accuracy revealed that 67.3%-76.7% of responses were correct but inadequate.
• A minority of responses were mixed (9.4%-11.2%) with incorrect or outdated information.

Conclusions:

• Chat GPT generally provides acceptable to good quality responses for IBD-related queries.
• Responses are predominantly correct but often lack comprehensiveness, requiring further optimization for clinical use.
• AI models like Chat GPT need refinement to minimize incorrect information for patient benefit.