Scene-dependent sound event detection based on multitask learning with deformable large kernel attention convolution.

Area of Science:

• Environmental sound analysis
• Machine learning
• Deep learning

Background:

• Sound event detection (SED) and acoustic scene classification (ASC) are related tasks in environmental sound analysis.
• Previous multitask learning (MTL) methods often use hard parameter-sharing, limiting feature exchange and information flow between SED and ASC.
• This hinders the ability to balance complex interrelationships and optimize performance for both tasks.

Purpose of the Study:

• To propose a novel multitask network for joint SED and ASC.
• To enhance sound event detection performance by utilizing acoustic scene information.
• To improve feature sharing and information flow between related audio analysis tasks.

Main Methods:

• Developed a residual multi-level feature extraction (R-MFE) framework for joint SED and ASC.
• Introduced the D-LKAC attention module, combining self-attention and convolution for global and local feature capture.
• Incorporated the MS-conv module to capture audio details from multiple dimensions, further enhancing SED.

Main Results:

• The proposed MTL method was evaluated on the TUT Acoustic Scenes 2016/2017 and TUT Sound Events 2016/2017 datasets.
• Experimental results demonstrated superior performance compared to state-of-the-art techniques.
• Achieved a significant improvement in F-scores by 6.44% for the joint tasks.

Conclusions:

• The novel R-MFE framework effectively addresses limitations of previous MTL approaches in environmental sound analysis.
• The proposed attention and convolutional modules enhance the model's ability to capture relevant audio features.
• The approach demonstrates a promising direction for improving joint sound event detection and acoustic scene classification.