Automated sentiment analysis in psychotherapy: Convergence with human-coded affective behavior and dyadic...

Emotional sentiment expressed during sessions is important in understanding psychotherapy processes and outcomes. Although human coding is typically considered the gold standard for assessing emotions, it is resource-intensive and difficult to scale. Automated sentiment analyses offer promising alternatives, yet little is known about (1) their convergence with human-coded affective behavior at the turn-level rather than session averages, (2) whether the association between automated sentiment and human-coded affect differs by the speaker (therapist vs. client), and (3) automated sentiment indicators' dyadic associations with early therapeutic alliance.

Using 108 individual early therapy sessions, trained coders coded affective behavior in sessions to generate turn-level positive, negative, and neutral affect scores. Additionally, to automatically quantify sentiment in transcripts at scale, we used a dictionary-based method (LIWC) and a natural language processing-based model (XLM-T).

Automated models showed nearly no convergence with human affect coding at the turn-level but limited convergence at the session level. Associations between automated sentiment and human-coded affect for negativity were stronger in clients' than therapists' turns. Automated sentiment was dyadically associated with therapist- and client-reported early alliance.

Automated sentiment may provide a scalable additional source of information for understanding the therapeutic process and therapeutic relationships.