Natural language processing models for predicting treatment outcomes in internet-delivered cognitive behavioral...

Area of Science:

• Psychiatry and Mental Health
• Computational Linguistics
• Health Informatics

Background:

• Internet-delivered Cognitive Behavioral Therapy (ICBT) effectiveness can be enhanced by predicting treatment outcomes.
• Patient-therapist communication via written messages is a key component of guided ICBT.
• Natural Language Processing (NLP) offers potential for analyzing these messages to predict outcomes, but evidence is limited.

Purpose of the Study:

• To investigate the predictive accuracy of NLP models for treatment outcomes in ICBT.
• To evaluate if NLP analysis of patient-therapist messages provides additional predictive value beyond symptom variables.

Main Methods:

• Trained three NLP models (TF-IDF, BERT, BELT) on 6613 patient-therapist messages from a 12-week ICBT program.
• Models were trained with and without initial symptom variables to predict post-treatment symptoms.
• Compared NLP models against a dummy model and a linear regression model using only symptom data. Employed multiple imputation and nested cross-validation.

Main Results:

• The linear regression model using only symptom variables performed best, outperforming NLP models.
• The BERT model showed slight improvement over the dummy model (RMSE 0.17 vs. 0.18).
• Combining BERT with symptom variables improved accuracy but not RMSE; symptom-only models achieved higher BACC (70%) and F1-score (0.66) than NLP models.

Conclusions:

• NLP analysis of patient-therapist messages in ICBT has limited predictive value for treatment outcomes.
• NLP did not add significant predictive value beyond using symptom variables alone.
• Further research is required to refine NLP methods for psychological treatment and assess text-based interaction potential in ICBT.