Evaluating the Performance of AI Large Language Models in Detecting Pediatric Medication Errors Across Languages: A...

Area of Science:

• Artificial Intelligence in Healthcare
• Pharmacovigilance
• Pediatric Drug Safety

Background:

• Medication errors pose a significant risk in pediatric pharmacotherapy.
• Evaluating the efficacy of artificial intelligence (AI) tools for medication error detection is crucial.

Purpose of the Study:

• To assess the performance of four AI models (GPT-5, GPT-4, Microsoft Copilot, Google Gemini) in identifying medication errors in pediatric case scenarios.
• To compare AI model performance across English and Arabic languages.

Main Methods:

• Sixty pediatric cases, half containing medication errors across four therapeutic systems, were analyzed.
• AI models were tested using a unified prompt in both English and Arabic.
• Performance metrics included accuracy, sensitivity, specificity, and reproducibility, analyzed using SPSS version 22.

Main Results:

• Microsoft Copilot achieved the highest accuracy (86.7% English, 85.0% Arabic), followed by GPT-5.
• Google Gemini exhibited the lowest accuracy (76.7% English, 73.3% Arabic).
• Arabic language performance was generally lower than English; Microsoft Copilot demonstrated superior reproducibility and inter-language agreement.

Conclusions:

• Microsoft Copilot outperformed other AI models in detecting pediatric medication errors in this study.
• The findings underscore the need for enhanced multilingual AI training to ensure equitable performance across languages.
• Human oversight and domain-specific AI training are vital for safe application in pediatric pharmacotherapy.