Comparative Analysis of Large Language Models in First-Aid Scenario Recognition and Management: An In Silico...

Large language models (LLMs) deliver real-time, conversational guidance, yet their reliability for time-critical first aid remains unclear.

Five standardized vignettes (drowning, animal bite, opioid overdose, lightning strike, and frostbite) were presented three times each to GPT-4o (OpenAI, San Francisco, CA, USA) and Claude 3.5 Sonnet (Anthropic, San Francisco, CA, USA). Outputs were scored (0 = incorrect/unsafe, 1 = incomplete, 2 = entirely correct) across six domains: diagnostic accuracy, first-aid advice, triage accuracy, comprehensiveness, safety, and consistency. Scores were averaged within and across vignettes.

Both LLMs achieved perfect diagnostic (2.0) and triage (2.0) scores. Claude 3.5 outperformed GPT-4o in first-aid accuracy (2.0 vs 1.5), comprehensiveness (1.5 vs 1.3), and consistency (2.0 vs 1.6). Safety ratings were comparable (1.9-2.0). Key GPT-4 omissions included naloxone administration for opioid overdose and immediate sheltering guidance after a lightning strike.

Claude 3.5 provided more complete and stable first-aid guidance than GPT-4, although both models reliably identified emergencies and advised on the appropriate escalation of care. Wider implementation warrants larger vignette sets, real-user simulations, and continuous monitoring for guideline concordance.