Related Experiment Video
Updated: Jan 21, 2026

Involving Individuals with Developmental Language Disorder and Their Parents/Carers in Research Priority Setting
Published on: June 6, 2020
Language-dependent performance of large language models in medical diagnostics: A comparative study of ChatGPT-4o and
El Habib El Mahdaoui1, Yaroslav O Mykhalko2
1UNIVERTITE DE MONTREAL, MONTREAL, CANADA; UZHHOROD NATIONAL UNIVERSITY, UZHHOROD, UKRAINE.
Objective:
Aim: The aim for this study was to evaluate the efficacy of ChatGPT-4o and Claude 3.5 Sonnet in diagnosing diseases from clinical case descriptions presented in various languages.
Patients And Methods:
Materials and Methods: The study used 50 clinical case descriptions provided by both LLMs in three languages - English, Arabic and Ukrainian. Diagnostic results were assessed by a panel of clinicians according to standard criteria, and statistical methods were employed to determine the significance of the difference between the obtained indicators.
Results:
Results: The overall diagnostic accuracy for ChatGPT-4o was 62.00%±15.62%, with performance significantly differing by language: 72.00%±10.00% in English, 44.00%±12.00% in Arabic, and 70.00%±9.00% in Ukrainian. A similar pattern was observed in Claude 3.5 Sonnet, which had a mean accuracy of 60.67%±14.50%. Both models performed significantly worse in Arabic compared to English and Ukrainian (p<0.05). Both models showed reduced, but significant, accuracy for cases published in 2024-2025 vs. 2023 (p>0.05).
Conclusion:
Conclusions: No significant differences in diagnostic accuracy were found between the LLMs; rather, the discrepancies were primarily related to the languages used. In multilingual healthcare settings, such as those involving Arabic, various underlying factors-particularly language complexity, cultural context, and the type of clinical cases-affect their performance. Further research is essential to improve the capabilities of LLMs, ensuring equitable healthcare access for diverse linguistically representative populations.
More Related Videos
Related Concept Videos
Language
Corballis and Suddendorf (2007) and Tomasello and Rakoczy (2003) highlight the role of language in...
Components of Language
Language Development
The critical period for language acquisition suggests that the ability to acquire language is at its peak early in life. As people age, this proficiency decreases. Language development begins very...
Language and Cognition
Higher Mental Functions of the Brain: Language
Language formation and comprehension take place in the dominant hemisphere. The dominant hemisphere is responsible for understanding the meaning of spoken, written, or sign language, as well as the ability to communicate. For most people, the left hemisphere is the dominant one. The right hemisphere, then, gives tone and emotional context to the...
Appendicitis-II: Diagnostic Studies and Management
Diagnosing Appendicitis
It requires a multifaceted approach, starting with a detailed physical examination to pinpoint the location and nature of the pain and identify any associated symptoms. Laboratory tests play a crucial role. A complete Blood Count (CBC) typically reveals leukocytosis (an increased number of...

