Applying a Transformer-based machine-learning model to classify caregiver and infant behaviours during dyadic...

Area of Science:

• Developmental Psychology
• Artificial Intelligence
• Behavioral Science

Background:

• Manual coding of caregiver-infant interactions is time-consuming and prone to bias.
• Multimodal interactions significantly impact infant development.
• Automated analysis offers potential for objective and efficient behavioral assessment.

Purpose of the Study:

• To develop and evaluate a multimodal machine-learning model for automatic detection of caregiver-infant behaviors.
• To assess the model's ability to generalize to unseen dyads and contexts.
• To identify challenges in applying AI to complex behavioral data.

Main Methods:

• Extracted audio, video, and pose features using AI models.
• Utilized a Transformer-based architecture for temporal pattern learning.
• Tested model performance on familiar versus entirely unseen caregiver-infant dyads.

Main Results:

• Achieved >98% accuracy when data from all dyads was included in training and testing.
• Performance dropped to ~55% when tested on completely unseen dyads.
• Indicated reliance on dyad-specific features rather than learned behaviors.

Conclusions:

• Current Transformer-based models face significant generalizability challenges with complex multimodal behavioral data.
• The study provides a foundation for improving AI models for behavioral analysis.
• Future research should focus on enhancing model robustness and applicability across diverse contexts.