Towards End-2-end Learning for Predicting Behavior Codes from Spoken Utterances in Psychotherapy Conversations.

Area of Science:

• Computational linguistics
• Speech processing
• Behavioral analysis

Background:

• Spoken language understanding typically requires complex pipelines, including Automatic Speech Recognition (ASR).
• Existing methods often depend on generating transcripts before analysis, adding computational steps and potential errors.
• Behavioral coding from speech is a key task in various fields, including psychotherapy.

Purpose of the Study:

• To develop a novel framework for predicting utterance-level labels directly from speech features.
• To eliminate the dependency on Automatic Speech Recognition (ASR) and transcription generation for behavioral coding.
• To enable transcription-free behavioral coding for spoken language analysis.

Main Methods:

• Utilized a pre-trained Speech-2-Vector encoder as a bottleneck to generate word-level speech representations.
• Employed an objective function similar to Word2Vec for the pre-trained encoder to learn speech feature encoding.
• Developed a classifier that uses only speech features and word segmentation information for predicting utterance-level labels.

Main Results:

• The proposed framework achieves competitive performance compared to state-of-the-art methods.
• The model successfully predicts psychotherapy-relevant behavior codes using only speech features.
• Demonstrated the efficacy of transcription-free behavioral coding.

Conclusions:

• The novel framework offers an efficient alternative to traditional ASR-dependent spoken language understanding pipelines.
• Directly predicting labels from speech features simplifies the process and reduces computational overhead.
• This approach holds significant potential for real-time and resource-constrained behavioral analysis applications.