Chat-based Artificial Intelligence model as a source of information for Inflammatory bowel disease

Artificial intelligence models could be a promising tool for providing scientific information to patients. The aim was to study the accuracy of the responses provided by Chat GPT (Generative Pre-trained Transformer tool) to common questions related to inflammatory bowel diseases (IBD).

Questions about broad domains of IBD (basic knowledge, symptomatology, diagnosis, therapeutics, prevention, nutrition, emotional, and social domains) were entered as new chat options in Chat GPT twice. The responses were assessed for quality (very poor, poor, acceptable, good, or very good) and accuracy (comprehensive, correct but inadequate, mixed - correct with incorrect/outdated data, and completely incorrect).

A total of 340 questions were included. The quality of responses in Chat GPT to overall questions in IBD was predominantly acceptable to good quality (82.4% in the first response and 78.2% in the second response). Poor quality responses constituted a minority (4.1% in the first response and 3.2% in the second response). The accuracy of responses to questions in all domains of IBD was correct but inadequate in 76.7% in the first response and 67.3% in the second response. Mixed correct with incorrect or outdated responses to questions in IBD were seen in 9.4% and 11.2% in the first responses and second responses, respectively. Each domain has variable quality and accuracy in responses.

Chat GPT predominantly provided correct but inadequate answers to common questions in IBD, with good to acceptable quality answers. The model could be optimized in ducing the incorrect responses for better use.