AI-based soundscape analysis: Jointly identifying sound sources and predicting annoyancea).

Area of Science:

• Environmental acoustics
• Artificial Intelligence
• Human-Computer Interaction

Background:

• Traditional soundscape studies rely on user surveys, limiting long-term monitoring and intervention assessment.
• Existing sound-signal approaches often focus on psycho-acoustic metrics or basic sound recognition, neglecting subjective appraisal.
• There's a need for AI-driven methods that can objectively analyze soundscapes, including both sound identification and human perception.

Purpose of the Study:

• To propose and evaluate an AI-based dual-branch convolutional neural network with cross-attention-based fusion (DCNN-CaF) for comprehensive soundscape characterization.
• To perform sound source classification (SSC) and human-perceived annoyance rating prediction (ARP) using the DeLTA dataset.
• To demonstrate the model's ability to capture the relationship between sound sources and perceived annoyance.

Main Methods:

• Developed a DCNN-CaF model integrating loudness and Mel features for enhanced soundscape analysis.
• Utilized the DeLTA dataset, which includes human-annotated sound source labels and perceived annoyance ratings.
• Compared the DCNN-CaF model against traditional machine learning methods and other AI models for SSC and ARP tasks.

Main Results:

• The DCNN-CaF model incorporating both loudness and Mel features outperformed models using only one feature for soundscape analysis.
• The proposed DCNN-CaF demonstrated superior performance in both SSC and ARP tasks compared to existing AI and traditional machine learning approaches.
• Correlation analysis confirmed that the AI model's prediction of annoyance aligns with human perception of sound sources.
• Generalization tests indicated the model's reliable performance even with previously unencountered sound sources.

Conclusions:

• The DCNN-CaF model offers a robust AI-driven solution for objective soundscape characterization, encompassing both sound recognition and appraisal.
• The model's ability to predict human annoyance demonstrates its potential for applications in environmental monitoring and urban planning.
• The findings support the use of AI for understanding complex human-environment interactions within sonic environments.