Towards sentiment aided dialogue policy learning for multi-intent conversations using hierarchical reinforcement...

Area of Science:

• Artificial Intelligence
• Natural Language Processing
• Human-Computer Interaction

Background:

• Developing virtual agents (VAs) for complex, multi-intent conversations is challenging.
• Existing systems often overlook user behavior and sentiment, focusing solely on semantics.
• User sentiment significantly impacts gratification and task success in dialogue systems.

Purpose of the Study:

• To introduce SentiVA, a novel dataset for sentiment-aided VA development in multi-intent conversations.
• To propose a Hierarchical Reinforcement Learning (HRL) approach for VAs handling multiple user intents.
• To integrate user sentiment into the policy learning of VAs for enhanced user adaptation.

Main Methods:

• Created the SentiVA dataset, annotated with intents, slots, and dialogue history sentiment.
• Developed an options-based Hierarchical Reinforcement Learning (HRL) framework for VAs.
• Incorporated both task success and sentiment-based rewards into the hierarchical value functions.

Main Results:

• Empirical results demonstrate that cumulative task-based and sentiment-based rewards are essential for success.
• Neither reward type alone is sufficient for optimal performance in multi-intent scenarios.
• The proposed HRL approach effectively learns strategies for managing multi-intent conversations.

Conclusions:

• User sentiment is crucial at every decision-making step for a fulfilling conversational experience.
• Integrating sentiment-based rewards into HRL is a novel and effective approach for VA development.
• This research paves the way for more user-adaptive and satisfying virtual agents.