Incorporating Multimodal Directional Interpersonal Synchrony into Empathetic Response Generation

Area of Science:

• Artificial Intelligence
• Computational Linguistics
• Affective Computing

Background:

• Interpersonal synchrony, the coordination between individuals, is crucial for effective human communication and empathy.
• Existing AI models for empathetic response generation often lack nuanced understanding of interpersonal dynamics.
• Modeling multimodal directional synchrony (positive and negative) offers a novel approach to enhance AI empathy.

Purpose of the Study:

• To investigate the impact of multimodal directional interpersonal synchrony on AI-generated empathetic responses.
• To propose and evaluate an AI model incorporating cosine similarity-based interpersonal synchrony.
• To analyze how positive and negative synchrony influence language and empathy levels in AI responses.

Main Methods:

• Development of an AI model integrating multimodal directional interpersonal synchrony (positive and negative) using cosine similarity.
• Comparative experiments evaluating models with and without synchrony features.
• Analysis of generated responses at linguistic and empathy levels.

Main Results:

• Models incorporating multimodal synchrony produced responses closer to ground truth and exhibited greater diversity.
• Positive synchrony was associated with enhanced emotional reactions, reduced exploration, and improved interpretation.
• Negative synchrony correlated with reduced exploration and increased interpretation in AI responses.

Conclusions:

• Multimodal directional interpersonal synchrony is a significant factor in improving AI empathetic response generation.
• The proposed model successfully integrates synchrony features, leading to more human-like and empathetic AI communication.
• Findings highlight the distinct roles of positive and negative synchrony in AI's emotional and cognitive processing for empathy.