Can a computer detect interpersonal skills? Using machine learning to scale up the Facilitative Interpersonal Skills...

Area of Science:

• Psychology
• Computer Science
• Artificial Intelligence

Background:

• Therapist interpersonal skills are crucial for effective psychotherapy.
• Current assessment methods are time-consuming and infrequent.
• Automated assessment tools are needed to improve efficiency and consistency.

Purpose of the Study:

• To evaluate the effectiveness of machine learning (ML) tools in automatically assessing therapist interpersonal skills.
• To determine if ML models can predict Facilitative Interpersonal Skills (FIS) scores.
• To compare ML assessment reliability with human raters.

Main Methods:

• Utilized data from 164 undergraduate students completing the Facilitative Interpersonal Skills (FIS) task.
• Employed an elastic net model with term frequency-inverse document frequency (TF-IDF) representation.
• Trained raters scored student responses to video vignettes depicting challenging psychotherapy moments.

Main Results:

• ML models predicted FIS total and item-level scores above chance (rho = .27–.53).
• Models achieved 31–60% of human rater reliability, explaining 13–24% of score variance.
• Performance was limited for skills relying heavily on vocal cues (e.g., verbal fluency, emotional expression).

Conclusions:

• Machine learning presents a viable approach for automating the assessment of interpersonal skills in psychotherapy.
• ML tools may be most effective with standardized stimuli and transcript data.
• Further research can refine ML models for more comprehensive assessment of therapist competencies.