Evidence-Grounded LLM Validation of MIMIC-IV ICD Labels

Automatically assigning ICD-10 diagnosis codes from discharge summaries is a central multi-label task in clinical NLP, yet widely used benchmarks such as MIMIC contain substantial label noise: many charted codes are not text-grounded in the note or are mis-specified. We present an LLM-based evidence-validation workflow that examines each (note, code) pair and: (1) determines whether the code is supported by the note, (2) extracts the corresponding evidence quote(s) from the note, and (3) when evidence exists but a more appropriate code can be inferred, suggests an evidence-based replacement. Applying this pipeline to 10,000 notes from the MIMIC-IV corpus, we derive two refined label sets: Evidence-Verified (EV), retaining only text-supported codes, and Evidence-Replaced (ER), substituting some existing codes in EV with better evidence-supported alternatives. We replicate six state-of-the-art ICD coding models (PLM-ICD, LAAT, MultiResCNN, CAML, Bi-GRU, CNN) under identical settings as Edin et al. and evaluate micro-precision, recall, and F1 using paired bootstrap resampling. Results show that removing unsupported charted codes from MIMIC substantially improves model performance and yields more trustworthy benchmarks for automated medical coding.