Multimodal affect analysis of psychodynamic play therapy

Area of Science:

• Psychology
• Computer Science
• Psychotherapy

Background:

• Accurate quantification of children's affective expressions is crucial in psychodynamic child psychotherapy.
• Existing methods for analyzing verbal and non-verbal cues can be time-consuming and subjective.

Purpose of the Study:

• To explore machine learning tools for automated facial and linguistic affect analysis.
• To enable easier, faster, and more precise quantification of children's affective expressions.

Main Methods:

• Utilized audio-video recordings from 148 therapy sessions with 53 Turkish children.
• Employed manual transcription and coding of affective expressions (pleasure, anger, sadness, anxiety) using the Children's Play Therapy Instrument (CPTI).
• Developed and combined facial and linguistic affect analysis modalities, using regression and machine learning techniques for prediction.

Main Results:

• Significant associations were found between automated affect predictions and CPTI dimensions (small to medium effect sizes).
• Fusion of facial and linguistic features yielded the best results for pleasure prediction.
• Linguistic analyses outperformed facial analyses for predicting anger, sadness, and anxiety.

Conclusions:

• The developed system aids in retrieving children's affective expressions.
• Further improvements are needed to enhance the precision of automated affect analysis.