Avatar-Based Picture Exchange Communication System Enhancing Joint Attention Training for Children With Autism

Children with Autism Spectrum Disorder (ASD) often struggle with social communication and feel anxious in interactive situations. The Picture Exchange Communication System (PECS) is commonly used to enhance basic communication skills in children with ASD, but it falls short in reducing social anxiety during therapist interactions and in keeping children engaged. This paper proposes the use of virtual character technology alongside PECS training to address these issues. By integrating a virtual avatar, children's communication skills and ability to express needs can be gradually improved. This approach also reduces anxiety and enhances the interactivity and attractiveness of the training. After conducting a T-test, it was found that PECS assisted by a virtual avatar significantly improves children's focus on activities and enhances their behavioral responsiveness. To address the problem of poor accuracy of gaze estimation in unconstrained environments, this study further developed a visual feature-based gaze estimation algorithm, the three-channel gaze network (TCG-Net). It utilizes binocular images to refine the gaze direction and infer the primary focus from facial images. Our focus was on enhancing gaze tracking accuracy in natural environments, crucial for evaluating and improving Joint Attention (JA) in children during interactive processes.TCG-Net achieved an angular error of 4.0 on the MPIIGaze dataset, 5.0 on the EyeDiap dataset, and 6.8 on the RT-Gene dataset, confirming the effectiveness of our approach in improving gaze accuracy and the quality of social interactions.