Experience with an Affective Robot Assistant for Children with Hearing Disabilities

Area of Science:

• Robotics
• Human-Computer Interaction
• Assistive Technology

Background:

• Children with hearing disabilities face challenges in audiological assessments.
• Traditional audiometry tests can be stressful and less engaging for pediatric patients.
• Emotion recognition technology offers potential for personalized and adaptive interventions.

Purpose of the Study:

• To develop and evaluate an assistive robotic system with emotion recognition for children with hearing disabilities during audiometry tests and rehabilitation.
• To compare the effectiveness of a robotic system combined with a tablet versus conventional and tablet-only setups.
• To analyze children's emotional responses and engagement levels during different testing scenarios.

Main Methods:

• Development of a system integrating a social humanoid robot (Pepper), an interactive interface, gamified audiometry, and an emotion recognition module using physiological signals (E4 wristband).
• Evaluation of three scenarios: conventional, tablet-only, and robot+tablet, with 16 children with hearing impairments (cochlear implant or hearing aid).
• Application of machine learning and deep learning models for classifying test setups and children's emotions (pleasant, neutral, unpleasant).

Main Results:

• Physiological signals collected during tests were successfully classified.
• Positive and negative emotions of children were better distinguished when interacting with the robot compared to other setups.
• The robot+tablet setup demonstrated enhanced engagement and emotional distinguishability.

Conclusions:

• The assistive robotic system with emotion recognition shows promise for improving audiometry and rehabilitation for children with hearing disabilities.
• Robot-assisted interventions can lead to more accurate assessment of children's emotional states during audiological tests.
• Further analysis of children's subjective evaluations and impressions of the robot's emotional behaviors is crucial for system refinement.