Integrating human and artificial intelligence for robust postmarketing safety surveillance systems: reflections from...

To describe considerations for integration of human and artificial intelligence for creating a postmarketing surveillance system capable of timely and reliably identifying causal effects of medications on safety endpoints.

The FDA has prioritized more extensive Electronic Health Records (EHR) integration along with generative artificial intelligence and machine learning (Gen AI/ML) into the national active surveillance program for medical products-the Sentinel Initiative. Based on our experience of leading these efforts, we provide perspectives on the opportunities and challenges of Gen AI/ML integration into Sentinel.

Using specific examples, we outline the role of Gen AI and ML in a causal inference framework for scalable information extraction, assessment of fitness-for-purpose of data sources, diagnosing residual confounding, and enhancing confounding adjustment. Critically, we outline steps and checkpoints along the way where human involvement remains indispensable.

In public health applications where stakes are high, use of Gen AI/ML needs to be carefully considered with appropriate guardrails ensuring human expert involvement.