Predicting dropout from psychological treatment using different machine learning algorithms, resampling methods, and...

Area of Science:

• Psychology
• Computer Science
• Health Informatics

Background:

• Dropout from psychological interventions leads to poor outcomes and high costs.
• Machine learning (ML) shows promise in predicting treatment dropout.
• Challenges in ML prediction include imbalanced datasets and small sample sizes.

Purpose of the Study:

• To enhance dropout prediction accuracy in psychological interventions.
• To compare ML algorithms, sample sizes, and resampling methods for dropout prediction.
• To identify optimal strategies for improving ML-based dropout prediction.

Main Methods:

• Evaluated twenty ML algorithms on twelve subsamples from a large dataset (N=49,602).
• Compared four resampling methods against no resampling.
• Assessed prediction accuracy using the F1-Measure on an independent holdout dataset.

Main Results:

• Resampling methods significantly improved ML algorithm performance for dropout prediction.
• Down-sampling emerged as a fast and accurate resampling technique.
• A minimum training sample size of N=300 cases was necessary for the highest mean F1-Score of .51.

Conclusions:

• Resampling methods can enhance the accuracy of predicting dropout in psychological interventions.
• Down-sampling is recommended due to its computational efficiency.
• Training datasets for dropout prediction should include at least 300 cases.