Using Machine Learning to Identify and Investigate Moderators of Alcohol Use Intervention Effects in Meta-Analyses

To illustrate a machine learning-based approach for identifying and investigating moderators of alcohol use intervention effects in aggregate-data meta-analysis.

We illustrated the machine learning technique of random forest modeling using data from an ongoing meta-analysis of brief substance use interventions implemented in general healthcare settings. A subset of 40 trials testing brief alcohol interventions (BAIs) was used; these trials provided 344 estimates of post-intervention effects on participants' alcohol use as well as data on 20 potential moderators of intervention effects. These candidate moderators included characteristics of trial methodology and implementation, intervention design and participant samples.

The best-fitting random forest model identified 10 important moderators from the pool of 20 candidate moderators. Meta-regression utilizing the selected moderators found that inclusion of prescriptive advice in a BAI session significantly moderated BAI effects on alcohol use. Observed effects were also significantly moderated by several methodological characteristics of trials, including the type of comparison group used, the overall level of attrition and the strategy used to address missing data. In a meta-regression model that included all candidate moderators, fewer coefficients were found to be significant, indicating that the use of a preliminary data reduction technique to identify only important moderators for inclusion in final analyses may have yielded improved statistical power to detect moderation.

Machine learning methods can be valuable tools for clarifying the influence of trial, intervention and sample characteristics on alcohol use intervention effects, in particular when numerous candidate moderators are available.