The Visual Predictive Check and Real-World Data

The visual predictive check (VPC) is a standard tool for assessing pharmacometric model suitability, producing visualizations that compare observed data with simulated data such that both model structure and model variability terms can be assessed. However, real-world data commonly reflect clinical decision-making that adapts therapy in response to patient data. As a result, VPCs constructed from such data may display apparent model misspecification, even when models are well-specified. To illustrate possible confounders, four common real-world therapy adaptation scenarios were simulated: (1) varying dose amount with constant interval, (2) varying dosing interval with constant dose amount, (3) patient dropout after identification of a suitable maintenance dose, and (4) variability in sample timing based on individual pharmacokinetic parameter estimates. For all scenarios, simulated observations were generated using the same model used to produce the VPC simulations, ensuring that the model was unbiased. When only the dose amount varied, the prediction-corrected VPC (pcVPC) appropriately indicated a well-specified model. In all other cases, both VPC and pcVPC erroneously suggested a misspecified model. These simulated results, together with our broader experience with complex real-world data, demonstrate that VPCs can be misleading when applied to real-world data if dosing interval, sample timing, or sample frequency varies in response to measured drug concentrations. We propose these four scenarios as benchmarks for evaluating the robustness of current and future model diagnostics to real-world data characteristics.