An agentic AI system enhances clinical detection of immune-related adverse events in a multi-phase validation study

Immune-related adverse events (irAEs) affect up to 40% of patients receiving immune checkpoint inhibitors, yet identification requires laborious, inconsistent manual chart review. Here we show that agentic artificial intelligence coupled with human verification could enhance efficiency, performance and consistency for irAE assessment. We developed and evaluated an agentic large language model system to extract the presence, temporality, severity grade, attribution and certainty of six irAE types from clinical notes. Retrospectively, the system achieved a macro-averaged F₁ of 0.92 for current-event detection and 0.66 for multi-class severity grading; self-consistency improved F₁ by 0.14. The best-performing configuration cost approximately US$0.02 per note. In prospective silent deployment over 3 months, the binary F₁ was 0.72-0.79. In a randomized crossover study of clinical trial staff, agentic assistance reduced annotation time by 2.71 min (P < 0.001), increased complete-match accuracy (odds ratio 1.45; 95% confidence interval 1.01-2.08; P = 0.045) and improved interannotator agreement (Krippendorff's α from 0.22-0.51 to 0.82-0.85).