Capturing Conversational Gestures for Embodied Conversational Agents Using an Optimized Kaneda-Lucas-Tomasi Tracker...

Area of Science:

• Human-Computer Interaction
• Artificial Intelligence
• Robotics

Background:

• Embodied conversational agents (ECAs) require synchronized verbal and non-verbal (gestural) communication for human-like interaction.
• Existing frameworks often lack realistic, diverse, and physically realized non-verbal expressions, limiting believability.
• The diversity and naturalness of gestures are critical for effective human-computer interaction.

Purpose of the Study:

• To propose and validate a method for automatically capturing and transforming real-world gestures into 3D representations for ECAs.
• To enhance the naturalness and expressiveness of ECA non-verbal behaviors.
• To improve the overall quality of human-computer interaction through realistic gestural communication.

Main Methods:

• Gesture capture from videos using the Kanade-Lucas-Tomasi (KLT) tracker.
• Signal processing with the Savitzky-Golay filter and kinematic modeling using a Denavit-Hartenberg (DH) based model.
• Integration within the EVA framework and objective evaluation using cosine similarity.

Main Results:

• Successful automatic capture and transformation of gestures into 3D representations for ECA motor skill repositories.
• Demonstrated naturalness in ECA gestures, leading to higher quality human-computer interaction.
• Achieved 96% similarity in synthesized movements based on objective cosine similarity evaluation.

Conclusions:

• The proposed method effectively addresses the challenge of diverse and natural gesture generation for ECAs.
• Automatic gesture capture and 3D representation significantly enhance the believability and quality of ECA interactions.
• Objective evaluation using cosine similarity provides a reliable measure of gesture synthesis accuracy.