ADPO: automatic-differentiation-assisted parametric optimization

Automatic differentiation (AD), a key method for accurately and efficiently computing derivatives in modern machine learning, is now implemented in Phoenix® NLME™ 8.6 for the first time and applied to the first-order conditional estimation extended least squares (FOCE ELS), Laplacian, and adaptive Gaussian quadrature (AGQ) algorithms. We name the AD implementation as 'automatic-differentiation-assisted parametric optimization' (ADPO), which can be enabled by checking the 'Fast Optimization' option. We present in detail how ADPO is implemented in the frequently used FOCE ELS algorithm, and analyze its performance from the benchmarks based on four PK/PD models. We show both ADPO and traditional FOCE ELS which uses gradients obtained from finite difference (FD) are reasonably accurate and robust, while the main advantage of ADPO being that it considerably reduces computation time no matter what ODE solvers are used: in general ADPO reduces the total run time by around 20% to 50% compared to traditional FOCE ELS. In a case for the realistic voriconazole model using 'auto-detect' ODE solver, 95% reduction in the total run time is observed.