A Tangible Multimodal Sensing Interface for Collaborative Task Engagement in Children With Autism Spectrum Disorder:...

Social engagement is critical for the developmental progress of children with autism spectrum disorder (ASD). While tangible technologies and serious games (SGs) show promise, a dual-purpose platform that both facilitates collaborative interaction and enables sensing of collaborative task engagement is required. Existing tangible interfaces often lack integrated sensing capabilities, while devices designed for complex input are typically not optimized for the sensory and interaction needs of this population.

This study aims to present InteractiSense, a tangible prototype designed to support collaborative play while enabling engagement estimation for children with ASD through embedded multimodal sensing. Our primary objectives were twofold: (1) to evaluate the prototype's preference, usability, and workload, and (2) to assess the feasibility of its embedded sensor data in generating valid inputs for preliminary machine learning (ML)-based engagement classification.

We developed InteractiSense, a spherical, soft-textured prototype embedded with 7 physio-behavioral sensors, alongside a collaborative SG promoting key social skills, and evaluated it against a familiar Xbox controller as a practical baseline. Fifteen children with ASD (mean age 11, SD 3 years; 10 males and 5 females) participated in a cross-sectional feasibility evaluation using a within-subjects mixed methods design. Preference data were analyzed using the McNemar test and Bayesian analysis, and usability and workload scores were compared using 2-tailed paired t tests. Qualitative caregiver interviews on InteractiSense and the SG were analyzed using thematic analysis, with device-related themes contextualizing quantitative findings during interpretation. To validate sensing feasibility, we analyzed the signal-to-noise ratio (SNR) of the sensor data and trained a preliminary extreme gradient boosting (XGBoost) model, comparing it with baseline models to classify binary engagement states using video-coded ground truths.

InteractiSense showed a tendency toward higher preference and comparable usability and workload metrics relative to the Xbox controller. Caregivers associated its round shape, soft texture, and simplified configuration with favorable perceptions and noted peer collaboration during the SG and interest in home use. Signal quality varied across sensing modalities: selected modalities provided analyzable data, whereas audio quality was low and pressure data were excluded from SNR and ML analyses. XGBoost achieved 83% (CI 0.76-0.90) balanced accuracy in binary engagement classification, exceeding random and majority-class baselines.

These findings provide initial evidence that InteractiSense can serve as a dual-purpose platform supporting collaborative play and engagement-aware sensing within the same tangible interface. Unlike prior systems focused on either interaction support or separate monitoring, InteractiSense combines embedded multimodal sensing with a social play interface in one platform for children with ASD. This study offers a proof-of-concept for extending tangible play systems toward engagement-related sensing and inference, with implications for future therapeutic and educational tools that monitor play and support adaptive intervention. However, feasibility was shown only at a preliminary level, requiring validation in larger cohorts and over longer observation periods.