NAAQA: A Neural Architecture for Acoustic Question Answering

Area of Science:

• Machine Learning
• Artificial Intelligence
• Signal Processing

Background:

• The Acoustic Question Answering (AQA) task aims to answer text questions about acoustic scenes, inspired by Visual Question Answering (VQA).
• Existing datasets and models face challenges with variable audio durations and sound variations between training and testing data.

Purpose of the Study:

• To introduce CLEAR2, a novel benchmark dataset for AQA that addresses specific challenges of acoustic inputs.
• To propose NAAQA, a neural architecture optimized for processing acoustic data and improving AQA performance.

Main Methods:

• Development of the CLEAR2 benchmark dataset, featuring variable duration scenes and distinct sound elements.
• Introduction of NAAQA, a neural network utilizing 1D convolutions in time and frequency on spectro-temporal representations.
• Inclusion of time coordinate maps to enhance temporal localization capabilities.

Main Results:

• NAAQAA achieves 79.5% accuracy on the AQA task, significantly outperforming previous VQA models with approximately four times fewer parameters.
• Time coordinate maps improved network performance by ~17 percentage points, while frequency coordinate maps had minimal impact.
• NAAQAA demonstrated strong performance on independent datasets (DAQA) and with the addition of a MALiMo module.

Conclusions:

• NAAQAA offers a computationally efficient and effective solution for the AQA task.
• The CLEAR2 benchmark provides a more robust evaluation for AQA systems.
• The release of CLEAR2 and NAAQA aims to stimulate further research in acoustic scene understanding.