Artificial Intelligence in the Trauma Bay: A Pilot Comparison With Surgical Trainees

BackgroundLarge language models (LLMs) have demonstrated strong performance on general medical knowledge assessments; however, their accuracy within high-acuity, guideline-driven surgical environments such as the trauma bay remains incompletely characterized.ObjectiveTo compare the accuracy of a contemporary LLM, Google Gemini, with junior general surgery residents on trauma knowledge questions derived from national practice management guidelines.MethodsThirty multiple-choice questions were developed from current trauma guidelines issued by nationally recognized professional organizations and independently validated by faculty trauma surgeons. Six junior general surgery residents (PGY-1-2) completed the assessment, generating 180 total responses. The LLM was tested on the same questions under standardized conditions. Accuracy was calculated with 95% confidence intervals and compared using a two-proportion z-test.ResultsResidents answered 157 of 180 questions correctly (87.2%, 95% CI 81.6-91.3). The LLM answered 27 of 30 questions correctly (90.0%, 95% CI 74.4-96.5). There was no statistically significant difference in accuracy between groups (P = .67).ConclusionIn this pilot study, a LLM demonstrated accuracy comparable to junior surgical residents when evaluated on trauma guideline-based questions. Although no significant difference was found, the findings of our exploratory study support cautious exploration of guideline-grounded artificial intelligence as an adjunct in surgical education while underscoring the need for broader validation. Further power studies are required to confirm these preliminary findings.