LLM-based pedagogical agent for ICU simulation instructor training: A quasi-experimental study

Intensive Care Unit (ICU) nursing is demanding, requiring advanced clinical decision-making and emergency management skills. Simulation-based instruction is central to ICU nursing education but remains constrained by the cost and time required for scenario authoring, limited faculty capacity for feedback, and slow content updates. Large language models (LLMs)-based pedagogical agents may augment instructor training by supporting rapid scenario generation, formative guidance, and on-demand assistance. However, evidence from real-world ICU instructor training is limited, and the balance between perceived benefits, usability, and objective educational outcomes is unclear.

To evaluate the feasibility and learner-perceived impact of integrating an LLM-based pedagogical agent into ICU simulation instructor training.

An exploratory quasi-experimental study was conducted with 40 ICU nurses from a tertiary hospital in February 2025. Participants were randomly assigned to an experimental group (n = 20) using the LLM-based AI teaching agent for simulation training, and a comparison group (n = 20) using traditional blended learning. The training effectiveness was assessed using the Chinese version of the Jeffries Simulation Design Scale (SDS), the System Usability Scale (SUS), the Adult Online Learning Self-Efficacy Scale, and a teaching satisfaction questionnaire. Data were analyzed using Wilcoxon rank-sum tests and t-tests.

The experimental group outperformed the comparison group in multiple areas. Specifically, in the SDS, the experimental group scored higher in case authenticity (5.00 vs. 4.00, p < 0.001), scenario complexity (5.00 vs. 4.00, p < 0.001), feedback mechanisms (5.00 vs. 4.00, p < 0.001), interactivity (5.00 vs. 4.00, p < 0.001), and teaching objectives (5.00 vs. 4.25, p < 0.001). The experimental group also showed higher self-efficacy in learning ability (16.0 vs. 13.0, p < 0.001) and learning technology (18.0 vs. 16.0, p = 0.045). Satisfaction was high in both groups and demonstrated a pronounced ceiling effect.

Embedding an LLM-based pedagogical agent into ICU simulation instructor training was feasible and associated with more favorable learner-perceived simulation design quality and online learning self-efficacy, while usability did not differ from traditional blended learning. Findings are preliminary and hypothesis-generating; future multi-centre, adequately powered randomized controlled trials are warranted to determine efficacy and isolate the LLM component's independent contribution.