Modelling learning dynamics in autism therapy through explainable multimodal representation learning

Area of Science:

• Computational neuroscience
• Developmental psychology
• Machine learning

Background:

• Autism Spectrum Disorder (ASD) behaviors are complex and challenging to quantify in clinical settings.
• Existing computational methods often require extensive data, lack interpretability, or raise privacy concerns.
• There is a need for unsupervised models that can analyze dynamic behaviors from minimally invasive inputs.

Purpose of the Study:

• To develop a privacy-preserving, unsupervised representation learning framework for analyzing ASD behaviors.
• To derive compact latent embeddings of behavior using skeletal pose and optical flow.
• To uncover the dynamic structure of behavior in children with ASD during therapy sessions.

Main Methods:

• Utilized a denoising temporal autoencoder on skeletal pose and optical flow features from the MMASD corpus.
• Trained the model on 255 multimodal windows from 15 child-clinician therapy sessions.
• Employed KL divergence to quantify intra-session adaptation and its correlation with social engagement.

Main Results:

• The unsupervised model identified a low-dimensional behavioral manifold with six latent motor clusters.
• Transition graphs revealed structured behavioral topologies, including hubs and bottlenecks.
• Saliency analyses pinpointed key anatomical and social features, with temporal saliency highlighting spontaneous gestures and social events. KL divergence correlated negatively with joint attention duration (r=-0.96, p=0.002).

Conclusions:

• Interpretable behavioral structures can be extracted from low-resolution, privacy-compliant inputs for Autism Spectrum Disorder (ASD) analysis.
• The framework shows potential for modeling learning dynamics, identifying motor patterns, and supporting objective progress tracking in ASD therapy.
• Future research should focus on clinical validation and application to larger datasets for generalizability and therapeutic utility assessment.