Effectiveness of ChatGPT for Clinical Scenario Generation: A Qualitative Study

A growing area is the use of ChatGPT in simulation-based learning, a widely recognized methodology in medical education. This study aimed to evaluate ChatGPT's ability to generate realistic simulation scenarios to assist faculty as a significant challenge in medical education.

This study employs a qualitative research design and thematic analysis to interpret expert opinions. The study was conducted in two phases. Scenario generation via ChatGPT and expert review for validation. We used ChatGPT (GPT-4) to create clinical scenarios on cardiovascular topics, including cardiogenic shock, postoperative cardiac tamponade after heart surgery, and heart failure. A panel of five experts, four nurses with expertise in emergency medicine and critical care and an anesthesia specialist, evaluated the scenarios. The experts' feedback, strengths and weaknesses, and proposed revisions from the expert discussions were analyzed via thematic analysis. Key themes and proposed revisions were identified, recorded, and compiled by the research team.

The clinical scenarios were produced by ChatGPT in less than 5 seconds per case. The thematic analysis identified six recurring themes in the experts' discussions: clinical accuracy, the clarity of learning objectives, the logical flow of patient cases, realism and feasibility, alignment with nursing competencies, and level of difficulty. All the experts agreed that the scenarios were realistic and followed clinical guidelines. However, they also identified several errors and areas that needed improvement. The experts identified and documented specific errors, incorrect recommendations, missing information, and inconsistencies with standard nursing practices.

It seems that, ChatGPT can be a valuable tool for developing clinical scenarios, but expert review and refinement are necessary to ensure the accuracy and alignment of the generated scenarios with clinical and educational standards.