Replicability, Reproducibility, and Agent-based Simulation of Interventions.

Area of Science:

• Computational epidemiology
• Health informatics
• Causal inference

Background:

• Secondary use of medical data and observational data for causal inference is increasing.
• Challenges include confounding variables and variations in medical practices across different settings.
• Lack of ground truth hinders evaluation of methods for addressing these challenges.

Purpose of the Study:

• To demonstrate agent-based simulation for evaluating causal inference methods with medical data.
• To explore the impact of bias, error, and site-specific variations on inference.
• To provide a framework for assessing new methods and sample size calculations for electronic health record (EHR) studies.

Main Methods:

• Agent-based simulation modeling of medical interventions and patient-provider interactions.
• Incorporation of patient mortality risks and provider-specific treatment effects (observed and latent).
• Creation of a simulated environment with known ground truth for method evaluation.

Main Results:

• Simulations can model complex interactions affecting inference.
• The approach allows for exploration of how bias and variation impact findings.
• Identifies potential causes for non-replication of findings across different sites.

Conclusions:

• Agent-based simulation offers a valuable tool for creating ground truth in medical data research.
• This method can enhance the evaluation of causal inference techniques.
• It aids in optimizing study design and sample size calculations for EHR-based research.