Affect Recognition in Hand-Object Interaction Using Object-Sensed Tactile and Kinematic Data

Area of Science:

• Human-Computer Interaction
• Affective Computing
• Robotics

Background:

• Understanding human affective states during object interaction is crucial for intuitive human-robot collaboration.
• Basic actions like grasping and rotating are fundamental to daily-life interactions and can convey emotional information.

Purpose of the Study:

• To investigate the recognition of human affective states (emotions and vitality forms) by processing object-sensed data.
• To evaluate the effectiveness of tactile and kinematics data from an object for emotion and action style classification.

Main Methods:

• Utilized the iCube, a 5 cm cube, to collect tactile maps and rotation data during basic actions.
• Conducted two studies: emotion classification (anger, sadness, excitement, gratitude) and vitality form classification (gentle vs. rude actions).
• Trained machine learning models using hand-crafted features and evaluated classifier performance.

Main Results:

• Emotion recognition achieved up to 82.7% accuracy. Notably, tactile data alone performed comparably to models using all 10 features.
• Vitality form classification differentiated gentle from rude actions with 84.85% accuracy.
• Confirmed that affective states and attitudes influence how individuals interact with objects.

Conclusions:

• Object-sensed tactile and kinematics data can effectively infer human affective states and action styles.
• Tactile sensing alone shows significant potential for emotion recognition in human-object interactions.
• This research provides a foundation for developing more perceptive and responsive interactive systems.