A Semantic Parsing Pipeline for Context-Dependent Question Answering over Temporally Structured Data

Area of Science:

• Computer Science
• Artificial Intelligence
• Human-Computer Interaction

Background:

• Traditional data exploration methods often lack intuitive interaction.
• Analyzing complex time-series data requires advanced querying capabilities.
• Integrating natural language with graphical interfaces can enhance user experience.

Purpose of the Study:

• To develop a hybrid question-answering system for time-series data.
• To enable users to query entities using both natural language and direct GUI interactions.
• To improve the understanding of entity states and behaviors through interactive exploration.

Main Methods:

• A pipeline combining speech recognition and semantic parsing.
• Utilizing Long Short-Term Memory (LSTM)-based architectures for speech transcription and semantic parsing.
• Implementing attention mechanisms and copying for context-dependent semantic parsing.
• Adapting pre-trained models for user-specific speech recognition.

Main Results:

• Both speech recognition and semantic parsing models significantly outperformed baseline methods.
• The full pipeline demonstrated robustness, with minimal accuracy loss due to speech recognition errors.
• The proposed system effectively handles context-dependent queries and interactions.

Conclusions:

• The developed question-answering paradigm enhances data presentation and navigation.
• This approach has significant potential for medical applications involving sensor data and life events.
• Hybrid natural language and GUI interaction offers a powerful new way to query complex datasets.