Related Experiment Videos
Reliability and readability of Artificial Intelligence generated information on acute cholecystitis: A comparative
Mabel Lucero Olarte Jurado1, Natalia Pimiento Blanco1, María José Prieto Otero1
1Facultad de Medicina, Universidad Autónoma de Bucaramanga, Bucaramanga, Colombia.
Digital Health
|August 13, 2026
Summary
AI language models show varying reliability and readability for acute cholecystitis (AC) information. Gemini performed best in English for reliability, while Perplexity was highest in Spanish, but readability varied significantly across models and languages.
Area of Science:
- Medical Informatics
- Artificial Intelligence in Healthcare
Background:
- Acute cholecystitis (AC) is a common emergency condition.
- AI language models offer potential for medical information synthesis.
- The reliability and readability of AI-generated medical content require evaluation.
Purpose of the Study:
- To assess the reliability of AI language models in providing information on acute cholecystitis.
- To evaluate the readability of AI-generated content across Spanish and English.
- To compare the performance of different AI models (ChatGPT, Gemini, Perplexity) in these aspects.
Main Methods:
- Standardized questions on AC were posed in Spanish and English.
- Reliability was assessed by independent reviewers using a validated tool.
- Readability was measured using Flesch-Szigrist (Spanish) and Flesch Reading Easy/Flesch-Kincaid (English) scores.
Main Results:
- In English, Gemini showed the highest reliability (85.71%), while Perplexity and ChatGPT scored 71.43%.
- In Spanish, Perplexity provided the most complete responses (57.14%), followed by ChatGPT and Gemini (42.86%).
- Significant differences in readability were found; Gemini was most readable in Spanish, ChatGPT in English (lowest Flesch-Kincaid grade).
Conclusions:
- AI models exhibit variable reliability and readability for acute cholecystitis information.
- Gemini excelled in English reliability, Perplexity in Spanish reliability.
- Perplexity consistently produced the least readable content in both languages, highlighting a need for improved AI medical information accuracy and accessibility.
