Two-Stage Voice Application Recommender System for Unhandled Utterances in Intelligent Personal Assistant

Area of Science:

• Natural Language Processing
• Human-Computer Interaction
• Recommender Systems

Background:

• Intelligent personal assistants (IPA) struggle with complex or ambiguous voice requests.
• Standard natural language understanding (NLU) limitations lead to poor user experiences when requests are unhandled.
• There is a need for improved methods to connect unhandled user utterances to relevant third-party voice applications (skills).

Purpose of the Study:

• To propose a novel two-stage recommender system for matching unhandled voice utterances to third-party skills.
• To address challenges in skill recommendation, including data bias and incomplete ground truth.
• To improve the user experience and functionality of intelligent personal assistants.

Main Methods:

• Developed a two-stage shortlister-reranker system to match unhandled utterances to skills.
• Employed a skill shortlister calculating lexical and semantic similarity between requests and skills.
• Utilized observed data from a baseline system, addressing exposure bias with two relabeling methods.

Main Results:

• The proposed system effectively retrieves candidate skills by assessing lexical and semantic similarity.
• Methods were developed to handle incomplete ground truth and mitigate exposure bias.
• Extensive offline experiments demonstrated the system's effectiveness.

Conclusions:

• The two-stage shortlister-reranker system significantly improves the matching of unhandled voice requests to relevant skills.
• The proposed methods effectively handle data limitations like incomplete ground truth and exposure bias.
• Online A/B testing confirmed a significant enhancement in user experience satisfaction.