Polyphonic Sound Event Detection Using Temporal-Frequency Attention and Feature Space Attention.

Area of Science:

• Machine Learning
• Audio Signal Processing
• Deep Learning

Background:

• Polyphonic sound classification is challenging due to discontinuous and unstable time-frequency variations.
• Traditional methods struggle with characterizing key features in complex sound events, leading to poor performance.
• Existing models often fail to effectively utilize temporal-frequency and feature space information.

Purpose of the Study:

• To propose a novel Convolutional Recurrent Neural Network (CRNN) model, TFFS-CRNN, for enhanced polyphonic sound event detection (SED).
• To improve the characterization of key feature information in polyphonic SED by integrating temporal-frequency and feature space attention mechanisms.
• To achieve superior classification performance and reduce error rates in polyphonic SED tasks.

Main Methods:

• Developed a TFFS-CRNN model integrating Log-Mel spectrograms and MFCCs as input features.
• Incorporated a temporal-frequency (TF) attention module to capture critical TF features.
• Implemented a feature space (FS) attention module for dynamic weighting of feature dimensions and a Bidirectional Gated Recurrent Unit (BGRU) for contextual learning.

Main Results:

• The TFFS-CRNN model demonstrated significant improvements on the DCASE 2016 and 2017 Task3 datasets.
• Achieved a 12.4% and 25.2% higher F1-score compared to winning systems in the DCASE challenges.
• Reduced the error rate (ER) by 0.41 and 0.37, respectively, indicating enhanced classification accuracy.

Conclusions:

• The proposed TFFS-CRNN model effectively addresses the challenges of polyphonic sound classification.
• The integration of TF and FS attention mechanisms enhances feature characterization and model performance.
• TFFS-CRNN offers superior classification performance and lower error rates for polyphonic sound event detection.