Automated Working Alliance Assessment in Psychological Counseling Using Gemini and XGBoost

Area of Science:

• Artificial Intelligence
• Clinical Psychology
• Natural Language Processing

Background:

• Machine learning for therapeutic alliance measurement is effective but limited by reliance on clean transcripts.
• Real-world counseling documentation is often heterogeneous and multilingual, posing challenges for existing models.
• Automated assessment models struggle with realistic documentation scenarios.

Purpose of the Study:

• To propose a framework for automated working alliance assessment from complex, multilingual case reports.
• To overcome limitations of current models in handling heterogeneous and multilingual counseling documentation.
• To enhance the applicability of automated assessment in realistic psychotherapy scenarios.

Main Methods:

• Fine-tuning language-specific BERT models for dialogue structuring and speaker role delineation.
• Utilizing Gemini-2.5-Flash for annotating dialogues with working alliance ratings.
• Developing a hybrid feature representation combining linguistic style and semantic content.
• Employing entropy-based mutual information analysis to identify informative linguistic features.
• Inputting extracted hybrid features into XGBoost for alliance assessment.

Main Results:

• The proposed framework demonstrates superior performance compared to state-of-the-art methods.
• The model exhibits strong generalization ability across different types of case reports.
• Effective processing of complex, multilingual reports for accurate alliance assessment was achieved.

Conclusions:

• The developed framework offers a robust solution for automated working alliance assessment in realistic, complex documentation.
• This approach significantly advances the application of machine learning in psychotherapy research and practice.
• The hybrid feature strategy and multilingual processing capabilities are key to the model's success.