Related Experiment Videos
Human-Centered Artificial Intelligence Framework for Overdose Mortality Surveillance, New York City, 2022‒2023
Karli R Hochstatter1, Talia Nadel1, Raavi Gupta1
1Karli R. Hochstatter, Talia Nadel, and Jan Gryczynski are with Friends Research Institute, Baltimore, MD. Raavi Gupta and Yuxin Li were with the Department of Computer Science, Columbia University, New York, NY, at the time of this study. Anubhav Jangra is with the Department of Computer Science, Columbia University. Nicole D'Anna, Jason Graham, and Hannah Johnson are with the New York City Office of Chief Medical Examiner, New York, NY. Jason Graham is with the New York City Office of Chief Medical Examiner, New York, NY. Nabila El-Bassel is with the School of Social Work, Columbia University, New York, NY. Ziqing Yang and Smaranda Muresan are with the Computer Science Department, Barnard College, Columbia University, New York, NY.
Abstract:
Objectives. To develop and evaluate human-centered artificial intelligence (AI) models that identify suspected overdose deaths using narrative reports from medicolegal death investigations and describe ethical considerations for using AI in overdose surveillance. Methods. We trained AI models using 2022-2023 data from the New York City Office of Chief Medical Examiner (OCME). Narrative death investigation reports (n = 9263) were analyzed using a 2-step pipeline that first extracted textual evidence for 18 expert-defined indicators of overdose and then predicted overdose using 2 large language model families: LLaMA-8B-Instruct and ClinicalModernBERT. Domain experts provided feedback throughout model development, and safeguards were implemented to protect confidentiality and evaluate algorithmic bias. Results. The LLaMA-8B-Instruct model achieved 88% positive predictive value (PPV), 92% sensitivity, 84% negative predictive value (NPV), 76% specificity, and 85% macro-F1. The ClinicalModernBERT model achieved 83% PPV, 92% sensitivity, 82% NPV, 67% specificity, and 81% macro-F1. Both AI models outperformed OCME's existing surveillance tool and conventional logistic regression approaches. Conclusions. Human-centered AI may support timely and accurate overdose surveillance, but responsible implementation requires expert oversight, a privacy-compliant infrastructure, and ongoing evaluation of potential bias. (Am J Public Health. 2026;116(10):1565-1574. https://doi.org/10.2105/AJPH.2026.308698).