Large Language Model-Based Identification of Acute Coronary Syndrome Management Delays

Acute coronary syndrome (ACS) requires prompt treatment, yet management delays are difficult to identify. In this study, we developed a large language model (LLM) system to identify ACS management delays and characterized delay cases. Admissions to internal medicine residents at NYU from July 2022 to June 2025 (n = 4,642) were included. Prompts were validated to determine if the resident admission note documented initiation of ACS management and if the initial cardiology consult note documented initiation of ACS management (ground truth) (n = 161 for each). Discordant cases were reviewed by three physicians using a validated tool to confirm management delays. Demographics and key clinical findings of patients with and without delays were compared. The LLM identified management delays with a 52% positive predictive value (n = 35/67). Patients who were older, females, and with preferred language other than English or Spanish were more likely to have a management delay (73.4 ± 15.3 vs 68.5 ± 12.6 years old, p = 0.036, 56.8% vs 34% females, p = 0.014, and 27.0% vs 15.5% other preferred language, p = 0.046, in management delay vs non-management delay cases). The management delay group had longer average time in hours to receiving heparin, aspirin, and cardiac catheterization (56.91 ± 56.78 vs 18.97 ± 13.76, p < 0.001, 13.94 ± 16.64 vs 8.23 ± 9.82, p = 0.005, and 65.12 ± 51.65 vs 39.51 ± 44.19, p = 0.006, respectively in management delay vs non-management delay cases). In conclusion, the LLM-based system we developed to identify ACS management delays can detect cases at scale to inform individual and systems-level interventions to improve quality of ACS care.