Reliability and readability assessment of appendicitis information from AI language models (ChatGPT, gemini and perplexity) in English and Spanish

Background and aimsAppendicitis is one of the most frequent conditions in emergency departments. Artificial intelligence (AI)-based language models represent a novel source of medical information; however, their reliability and readability across languages remain uncertain. This study aimed to evaluate the reliability and readability of AI-generated appendicitis information.MethodsSeven standardised questions were formulated in Spanish and English. Two independent reviewers assessed response reliability using a validated tool. Spanish readability was measured using the Szigriszt-Pazos formula, while English readability was assessed via the Flesch Reading Ease (FRE) and Flesch-Kincaid scores. Results were compared across tools and languages.ResultsFor Spanish reliability, Perplexity® produced the highest proportion of complete responses (71.43%), followed by ChatGPT® (57.14%) and Gemini® (28.57%). In English, Perplexity® and Gemini® both achieved 71.43%, followed by ChatGPT® (57.14%). Readability analysis via FRE formula in English revealed significant differences between the models (p < 0.05). Perplexity® showed the best reliability but yielded text that was difficult to read in Spanish and very difficult in English.ConclusionPerplexity® emerged as the most reliable model in both languages, although readability remains a concern. There is a clear need for improvements to optimize the accuracy and accessibility of AI-generated medical information.