The Potential and Pitfalls of ChatGPT in Toxicological Emergencies

Poisoning cases involve a wide variety of toxic agents and remain a significant concern for emergency departments. Rapid and accurate intervention is crucial in these cases; however, emergency physicians often face challenges in accessing and applying up-to-date toxicology information in a timely manner. ChatGPT, an AI language model, shows promise as a diagnostic aid in healthcare settings, offering potentially valuable support in the management of toxicological emergencies.

In this study, we aimed to evaluate the potential of ChatGPT in answering toxicology study guide questions, simulating its utility as a decision-support tool.

This study involves an evaluation of ChatGPT's performance in answering toxicology study guide questions from the Study Guide for Goldfrank's Toxicologic Emergencies, designed to simulate its utility as a decision-support tool in toxicological emergencies. ChatGPT's responses were compared with the accuracy rates of responses from medical trainees using the same toxicology study guide questions. This accuracy rate is categorized as human response.

ChatGPT correctly answered 89% of the toxicology questions, outperforming human responders, who had a mean accuracy rate of 56%. However, ChatGPT was less accurate in responding to pediatric and complex case-based questions, highlighting areas where AI models may require further refinement.

The study suggests that ChatGPT has substantial potential as an assistive tool for emergency physicians managing toxicological emergencies, particularly in high-stress and fast-paced environments. Despite its strong performance, the AI model's limitations in handling specific clinical scenarios indicate the need for continuous improvement and careful application in medical practice.