Related Experiment Video
Updated: May 4, 2026
Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness
Published on: December 6, 2024
Comparative performance of artificial intelligence models in intensive care nursing questions: an evaluation of
Seçil Gülhan Güner1, Ziya Tan2, Serhat Gülpınar3
1Department of Internal Medicine Nursing, Faculty of Health Sciences, Karadeniz Technical University, Üniversite Neighbourhood, Farabi Street, Number: 88, Ortahisar , Trabzon, 61080, Türkiye. secilgulhan@ktu.edu.tr.
Background:
The complexity and rapidly evolving nature of critical patient care in Intensive Care Units underscore the importance of the accuracy and timeliness of nursing decisions, further highlighting the significance of nursing education. This study aims to examine the accuracy of four generative artificial intelligence tools (ChatGPT 5.0 Plus, ChatGPT 5.0, DeepSeek, and Google Gemini) in answering multiple-choice questions related to the intensive care nursing exam, a fundamental area in nursing education.
Methods:
In the study, the ChatGPT 5.0 Plus, ChatGPT 5.0, DeepSeek, and Google Gemini models were evaluated using a test data set consisting of 55 questions. The questions were classified according to their difficulty levels as easy (n = 16), medium (n = 17), and difficult (n = 22). The models' correct response rates and standard or unique correct/incorrect response distributions were examined. Computer-assisted statistical analysis used the Chi-square, one-way ANOVA, and Post-hoc Tukey tests. The study was reported according to STROBE.
Results:
According to the study results, the success rates of all models were similar for easy and medium-level questions (70-82%), and the difference between them was not statistically significant (p > 0.05). Under difficult questions, however, the performance of the models diverged significantly, with Google Gemini achieving the highest success rate at 77.27% and DeepSeek showing the lowest performance at 45.45%. The chi-square analysis revealed no statistically significant difference in the correct/incorrect distribution among the models (χ²=3.69; p = 0.296), but at the observational level, Google Gemini had a higher number of unique correct answers (n = 6) compared to the other models. ChatGPT 5.0 was found to have no unique errors.
Conclusion:
In conclusion, while AI models generally showed similar levels of success in intensive care nursing exam questions, Google Gemini demonstrated superior performance in difficult questions, and DeepSeek showed the lowest level of success among the models. The study provides an essential comparative framework regarding the usability of AI-based learning and assessment tools in nursing education. It offers guidance for the future development of AI-based educational technologies.
Clinical Trial Number:
Not applicable.
