Learning to generate pointing gestures in situated embodied conversational agents.

Area of Science:

• Robotics
• Human-Computer Interaction
• Artificial Intelligence

Background:

• Effective human-robot communication requires both verbal and non-verbal expression.
• Current research often prioritizes verbal communication, neglecting non-verbal cues vital for situated interactions.
• Non-verbal communication, like gestures, is crucial for agents to adapt flexible interaction strategies.

Purpose of the Study:

• To develop a system for learning non-verbal communicative gestures in embodied interactive agents.
• To enhance agent communication capabilities in physically situated settings through gesture generation.
• To improve both the naturalness and referential accuracy of agent-generated gestures.

Main Methods:

• Utilized a combination of imitation learning and reinforcement learning within a physically simulated environment.
• Trained an agent to generate pointing gestures for referential tasks.
• Evaluated the system against baseline models using subjective (virtual reality) and objective measures.

Main Results:

• The proposed system achieved high motion naturalness and high referential accuracy in gesture generation.
• Subjective evaluations in a virtual reality referential game demonstrated superior performance compared to a supervised learning baseline.
• Objective evaluations confirmed the model's effectiveness in a simulated physical environment.

Conclusions:

• Combining imitation and reinforcement learning is a promising approach for generating communicative gestures in embodied agents.
• The developed system shows robustness in simulated physical environments, indicating potential for real-world robot applications.
• The research highlights the importance of non-verbal communication for enhancing human-agent interaction and situated intelligence.