A new equivalence based metric for predictive check to qualify mixed-effects models

Area of Science:

• Pharmacometrics
• Computational Biology
• Drug Development Modeling

Background:

• Effective decision-making in drug development relies on robust modeling.
• Model qualification is crucial for ensuring the reliability of predictive models.

Purpose of the Study:

• To evaluate predictive checks as a covariate model qualification technique.
• To introduce and assess alternative criteria for qualifying drug development models.

Main Methods:

• Simulated concentration-time profiles for an intravenous drug in male and female subjects.
• Employed predictive checks with discrepancy and test statistics to compare true and false models.
• Evaluated qualification criteria including predictive p-value (Pp), probability of equivalence (peqv), and Kolmogorov-Smirnov test (pks).

Main Results:

• Predictive p-values (Pp) showed uniform distribution, making false model qualification unlikely.
• Probability of equivalence (peqv) indicated poor predictive performance for the false model compared to the true model at later time points.
• Kolmogorov-Smirnov test (pks) did not differentiate between true and false models.
• Test statistics, particularly equivalence-based comparisons, were more effective than discrepancy variables in rejecting false models.

Conclusions:

• Equivalence-based comparisons using test statistics offer more informative model qualification than significance-based approaches.
• While predictive checks may not offer routine advantages over existing methods, they can provide insights when models are used for trial design.
• Models intended for trial design should demonstrate the ability to regenerate the data used for their construction.