Automatic detection and labeling of self-stimulatory behavioral patterns in children with Autism Spectrum Disorder

An infrastructure to record, detect and label the behavioral patterns of children with Autism Spectrum Disorder (ASD) has been developed. The system incorporates 2 different sensor platforms which are wearable and static. The wearable system is based on accelerometer which detects behavioral patterns of a subject, while the static sensors are microphones and cameras which captures the sounds, images and videos of the subjects within a room. The video also provides ground truth for wearable sensor data analysis. The system labels the segment of video data upon detection of the autistic behavior. That is, it stores the time of the video when the activities are detected. Time-Frequency methods are used to extract features and Hidden Markov Model (HMM) are used for analyzing the accelerometer signal. Using these methods, we are able to achieve 91.5% of classification rate for behavioral patterns studied in this paper which is used to label and save data.