Development of a mixture model allowing for smoothing functions of longitudinal trajectories

In the health and social sciences, two types of mixture models have been widely used by researchers to identify participants within a population with heterogeneous longitudinal trajectories: latent class growth analysis and the growth mixture model. Both methods parametrically model trajectories of individuals, and capture latent trajectory classes, using an expectation-maximization algorithm. However, parametric modeling of trajectories using polynomial functions or monotonic spline functions results in limited flexibility for modeling trajectories; as a result, group membership may not be classified accurately due to model misspecification. In this paper, we propose a smoothing mixture model allowing for smoothing functions of trajectories using a modified algorithm in the M step. Specifically, participants are reassigned to only one group for which the estimated trajectory is the most similar to the observed one; trajectories are fitted using generalized additive mixed models with smoothing functions of time within each of the resulting subsamples. The smoothing mixture model is straightforward to implement using the recently released "gamm4" package (version 0.2-6) in R 3.5.0. It can incorporate time-varying covariates and be applied to longitudinal data with any exponential family distribution, e.g., normal, Bernoulli, and Poisson. Simulation results show favorable performance of the smoothing mixture model, when compared to latent class growth analysis and growth mixture model, in recovering highly flexible trajectories. The proposed method is illustrated by its application to body mass index data on individuals followed from adolescence to young adulthood and its relationship with incidence of cardiometabolic disease.