Leveraging large language models to identify microcounseling skills in psychotherapy transcripts

Area of Science:

• Psychology
• Artificial Intelligence
• Computational Linguistics

Background:

• Microcounseling skills are crucial for effective psychotherapy.
• Manual coding of these skills is labor-intensive and costly.
• Automated methods are needed to analyze therapy session transcripts.

Purpose of the Study:

• To investigate the efficacy of large language models (LLMs) in automating microcounseling skill identification.
• To fine-tune GPT-4.1 for classifying therapist utterances and assessing microcounseling skills.
• To evaluate the performance of LLMs in analyzing psychotherapy session transcripts.

Main Methods:

• Fine-tuning GPT-4.1 on human-annotated psychotherapy transcripts.
• Developing a pipeline for transcript preprocessing, dialogue segmentation, and supervised fine-tuning.
• Training the model to classify therapist utterances, generate explanations, and suggest alternative responses.

Main Results:

• The fine-tuned GPT-4.1 model achieved strong performance metrics (Accuracy: 0.78, F1: 0.78).
• The model successfully identified common and structurally distinct microcounseling skills.
• Performance was limited in detecting nuanced skills requiring understanding of implicit relational dynamics.

Conclusions:

• Fine-tuned LLMs demonstrate significant potential for automated coding of microcounseling skills in psychotherapy.
• This technology offers a scalable solution to enhance psychotherapy research and clinical practice.
• Further development is needed to address limitations in identifying subtle, context-dependent therapeutic skills.