ConsultCraft: Reimagining Perioperative Education With AI-Based Adaptive Case Discussions

The American Medical Association's vision of precision education requires personalized, scalable learning tools. Current medical education approaches lack systems that integrate data analytics with efficient content delivery. Large language models (LLMs) are a promising approach but risk propagating misinformation without expert oversight.

We developed ConsultCraft, a web-based perioperative case discussion application using the GPT-4o application program interface to reference cases developed using Claude 3.5 Sonnet with expert revision of the case narrative and learning points. Users engaged in simulated clinical case discussions designed to enhance critical thinking skills in clinical decision making. They could choose between immediate feedback (tutor mode) or deferred feedback (immersion mode). We analyzed transcripts over a 4-week period for question progression and feedback accuracy, including inferences, false credit given, and false knowledge gaps implied.

We analyzed 49 sessions across 9 cases with 80% in tutor mode. The LLM asked users 306 questions and generated 236 feedback responses. Analysis revealed a low rate of potentially misleading inferences (2.1%) with false credit given in 4.2% of feedback responses and false knowledge gaps implied in 9.7%.

ConsultCraft successfully combines LLM language processing capabilities with expert content to create personalized case discussions that minimize the potential for propagating misleading information. Future directions include content mapping to certification requirements and International Classification of Diseases, 10th Revision, codes and integration with assessment and scheduling data to target gaps in knowledge and experience. This approach demonstrates how educators can leverage augmented intelligence to create tailored educational tools that preserve expert guidance while achieving the accessibility and adaptability required for precision education.