Related Experiment Video
Updated: Jul 6, 2025
12:08
Improving IV Insulin Administration in a Community Hospital
Published on: June 11, 2012
18.8K
ChatGPT-Exploring Its Role in Clinical Chemistry
Ridwan B Ibrahim1,2, Anil K Chokkalla1,2, Kaitlyn Levett1
1Department of Pathology and Immunology, Baylor College of Medicine, Houston, TX, USA.
Annals of Clinical and Laboratory Science
|January 5, 2024
Summary
Artificial intelligence language models like ChatGPT 3.5 and GPT-4 are less accurate than clinical chemists in answering clinical chemistry questions. Human experts remain essential for reliable laboratory test result interpretation.
Area of Science:
- Clinical Chemistry
- Artificial Intelligence in Healthcare
- Medical Diagnostics
Background:
- Artificial intelligence (AI) language models are increasingly integrated into various professional fields.
- The utility of advanced AI models like ChatGPT 3.5 and GPT-4 in specialized medical domains requires thorough evaluation.
- Clinical chemistry relies on accurate interpretation of laboratory tests for patient diagnosis and management.
Purpose of the Study:
- To compare the accuracy of AI language models (ChatGPT 3.5 and GPT-4) against human experts in answering clinical chemistry questions.
- To assess the performance of AI models relative to clinical chemistry trainees and faculty.
- To determine the current capabilities and limitations of AI in clinical chemistry consultations.
Main Methods:
- A set of 35 clinical chemistry questions were curated from real-world scenarios.
- ChatGPT 3.5, GPT-4, clinical chemistry residents, fellows, and faculty were tasked with answering these questions.
- Responses were evaluated for accuracy and scored based on the respondent's experience level.
Main Results:
- Senior clinical chemistry faculty achieved 100% accuracy, outperforming junior faculty (90.5%), fellows (82.9%), and residents (71.4%).
- Both ChatGPT 3.5 (60%) and GPT-4 (71.4%) demonstrated lower accuracy rates than all human participant groups.
- GPT-4 showed improvement over ChatGPT 3.5, matching resident performance, but both AI models were significantly outperformed by experienced chemists.
Conclusions:
- Current AI language models, including ChatGPT 3.5 and GPT-4, are not yet capable of replacing trained clinical pathologists for answering complex clinical chemistry questions.
- Users without specialized training in clinical chemistry should exercise extreme caution when interpreting laboratory results using AI chatbots.
- Human expertise remains critical for ensuring the accurate and safe application of clinical chemistry diagnostics.
