The use of the EM algorithm for regularization problems in high-dimensional linear mixed-effects models

The expectation-maximization (EM) algorithm is a popular tool for maximum likelihood estimation, but its use in high-dimensional regularization problems in linear mixed-effects models has been limited. In this article, we introduce the EMLMLasso algorithm, which combines the EM algorithm with the popular and efficient R package glmnet for Lasso variable selection of fixed effects in linear mixed-effects models and allows for automatic selection of the tuning parameter. A comprehensive performance evaluation is conducted, comparing the proposed EMLMLasso algorithm against two existing algorithms implemented in the R packages glmmLasso and splmm. In both simulated and real-world applications analyzed, our algorithm showed robustness and effectiveness in variable selection, including cases where the number of predictors $\left(p\right)$ is greater than the number of independent observations $\left(n\right)$. In most evaluated scenarios, the EMLMLasso algorithm consistently outperformed both glmmLasso and splmm. The proposed method is quite general and simple to implement, allowing for extensions based on ridge and elastic net penalties in linear mixed-effects models.