Generating Real-World Evidence From AI Deployed in Health Systems

Health systems increasingly deploy predictive and generative artificial intelligence (AI) to improve care delivery and outcomes. Predictive AI models estimate the risk of future events to inform targeted interventions, while generative AI systems extract, summarize, and communicate information and may automate selected tasks. Routine deployment generates clinical, operational, and interaction data that can support evaluation of the effectiveness and safety of AI-enabled workflows.

We review approaches to generating real-world evidence (RWE) from routine deployment of predictive and generative AI across four implementation configurations: silent deployment, human-in-the-loop (HITL), human-on-the-loop (HOTL), and automated. For each configuration, we describe the evaluation questions, routinely generated data, and feasible study designs.

Each implementation configuration supports different forms of evidence about model performance, workflow effectiveness, human oversight, and safety. For both predictive and generative AI, silent deployment supports assessment of local performance and subgroup differences. Once predictive AI drives action, evaluating the effectiveness of an implemented workflow (HITL, HOTL, or automated) generally requires causal inference approaches. For generative AI, HITL implementations can generate data on edits, overrides, time on task, usability, cognitive burden, and safety-relevant failure modes. HOTL and automated generative AI implementations can generate data on escalation performance, human follow-up work, downstream outcomes, and safety signals.

AI implementation should be paired with methods for generating RWE on the effectiveness and safety of AI-enabled workflows. In HOTL and automated configurations, escalation performance and residual human workload become particularly important components of evaluation.