Novel optimization techniques for inferring heterogeneous population dynamics

In this paper, we introduce a new optimization algorithm that is well suited to solve parameter estimation problems that arise when inferring heterogeneous population dynamics. In these estimation problems, parameter estimation is complicated by the presence of two types of constraints: inequality constraints (e.g., non-negativity and boundedness of rates (so-called box-constraints)) and equality constraints that arise due to the need of the population fractions to sum to one. We call our new method cubic regularized Newton with affine scaling (CRNAS). In contrast to so-called first-order methods, which solely rely on the gradient of the objective function, our method utilizes the Hessian of the objective. As a result, it is able to focus on points that satisfy the second-order optimality conditions, as opposed to first-order methods that simply converge to critical points. This is an important feature in parameter estimation problems, where the objective function is often non-convex; as a result, there can be many critical points, which makes it nearly impossible to identify the global minimum. We use an affine scaling approach to handle a wide class of constraints, including equality constraints. We establish that CRNAS identifies a point that satisfies $ \epsilon $-approximate second-order optimality conditions within $ O(\epsilon^{-3/2}) $ iterations. Finally, we compare CRNAS with MATLAB's optimization solver fmincon on three different test problems. These test problems all feature mixtures of heterogeneous populations, a problem setting that CRNAS is particularly well-suited for. Our numerical simulations show that CRNAS has a favorable performance, thereby performing comparable, if not better than, fmincon in accuracy and computational cost for most of our examples.