Modelling learning dynamics in autism therapy through explainable multimodal representation learning

Autism Spectrum Disorder (ASD) presents with complex, temporally evolving motor and social behaviours that are difficult to quantify in ecologically valid clinical contexts. While recent computational methods offer diagnostic insights, many depend on fully supervised learning, high-resolution video, or artificial experimental constraints, limiting scalability, interpretability, and privacy compliance. Few approaches leverage unsupervised models to uncover dynamic behavioural structure from minimally invasive inputs.

To address these limitations, we propose a privacy-preserving, unsupervised representation learning framework that operates solely on skeletal pose and optical flow features. Using 255 multimodal windows from 15 therapy sessions in the MMASD corpus, a publicly available, privacy-safe dataset of child-clinician interactions, we train a denoising temporal autoencoder to derive compact latent embeddings of behaviour.

The model uncovers a low-dimensional behavioural manifold composed of six latent motor clusters. Transition graphs reveal structured topologies, including behavioural hubs and bottlenecks. Saliency analyses identify anatomically and socially relevant features, such as joint pairs (LWrist-LAnkle, Neck-Rear Head) and dynamic flow regions (e.g., index pair 7, 14). Temporal saliency, based on reconstruction error, highlights spontaneous gesture onsets and socially salient events. KL divergence between early and late session phases quantified intra-session adaptation (range: 0.06-17.7) and showed a strong negative correlation with joint attention duration (r = -0.96, p = 0.002), suggesting links between behavioural dynamics and social engagement.

These findings offer preliminary evidence that interpretable behavioural structure can be extracted from low-resolution, privacy-compliant inputs. While based on a limited sample, the framework illustrates potential for modeling learning dynamics, identifying salient motor patterns, and supporting objective progress tracking in ASD therapy. Future work will involve clinical validation and application to larger, longitudinal datasets to assess generalizability and therapeutic utility.