Visual and Thermal Image Processing for Facial Specific Landmark Detection to Infer Emotions in a Child-Robot...

Area of Science:

• Robotics
• Computer Vision
• Affective Computing

Background:

• Child-Robot Interaction (CRI) is a growing research area.
• Accurate emotion recognition is crucial for effective CRI.
• Existing methods may lack precision in dynamic interaction environments.

Purpose of the Study:

• To develop an enhanced emotion recognition system for children during CRI.
• To improve the accuracy of facial region localization using combined imaging techniques.
• To enable social robots to better understand and respond to children's emotional states.

Main Methods:

• Utilized both visual (RGB) and Infrared Thermal Imaging (IRTI) cameras.
• Employed Viola-Jones algorithm for facial region detection and homography for image plane transfer.
• Implemented a novel ROI selection method based on expert criteria and error probability.
• Applied Principal Component Analysis (PCA) for dimensionality reduction and Linear Discriminant Analysis (LDA) for classification.

Main Results:

• The proposed ROI localization method demonstrated significantly lower errors compared to traditional Viola-Jones.
• The system achieved a mean accuracy of 85.75% and Kappa value of 81.84% for recognizing five basic emotions (disgust, fear, happiness, sadness, surprise).
• The system successfully inferred child emotions during real-time child-robot interaction using thermal imaging data.

Conclusions:

• The developed system offers a robust and accurate method for child emotion recognition in CRI.
• The novel ROI selection technique enhances the reliability of facial landmark tracking.
• This technology can be integrated into social robots to foster more empathetic and responsive interactions with children.