How do you feel? Using natural language processing to automatically rate emotion in psychotherapy

Area of Science:

• Psychology
• Computational Linguistics
• Artificial Intelligence

Background:

• Emotional distress is a primary driver for seeking psychotherapy.
• Analyzing emotional exchange in therapy is crucial but traditionally limited by scale and methodology.
• Existing methods like observer ratings or dictionary-based text analysis lack contextual understanding and efficiency.

Purpose of the Study:

• To evaluate the effectiveness of advanced Natural Language Processing (NLP) models for analyzing emotional content in psychotherapy.
• To compare a new BERT-based NLP model against a previous sentiment model and a dictionary-based model (LIWC).
• To assess the scalability and accuracy of automated emotion detection in large-scale psychotherapy transcript data.

Main Methods:

• Utilized a database of 97,497 psychotherapy utterances.
• Trained a BERT (Bidirectional Encoder Representations from Transformers) model using human ratings of emotion.
• Compared the performance of the BERT model against a unigram sentiment model and the LIWC (Linguistic Inquiry and Word Count) dictionary-based model.

Main Results:

• The BERT model achieved the highest accuracy in detecting emotion, with a kappa score of 0.48.
• The unigram sentiment model (kappa = 0.31) outperformed the LIWC model (kappa = 0.25).
• Advanced NLP, specifically BERT, demonstrates superior performance over traditional methods for sentiment analysis in psychotherapy.

Conclusions:

• Machine learning, particularly NLP models like BERT, offers a scalable and accurate approach to analyzing emotional dynamics in psychotherapy.
• This technology can overcome the limitations of manual analysis and traditional sentiment analysis tools.
• Future research can leverage these advanced NLP techniques for large-scale studies on therapeutic alliance and treatment outcomes.