CABIF-Net: Robust Confidence-Based Audio-Visual Fusion for Fine-Grained Bird Recognition

Area of Science:

• Ecology
• Computer Science
• Bioacoustics

Background:

• Fine-grained bird identification is vital for ecological research but faces challenges from noise and inconsistent data quality.
• Existing methods struggle with heterogeneous audio-visual data, limiting real-world application in monitoring and conservation.

Purpose of the Study:

• To develop an advanced audio-visual feature fusion framework for robust fine-grained bird classification.
• To address challenges of modality imbalance and background noise in real-world bird identification scenarios.

Main Methods:

• Proposed CABIF-Net framework with a confidence-based Top-K mean pooling module for optimizing visual representations.
• Implemented a Confidence Calibration module for dynamic modality reliability assessment and Bidirectional Inter-modulation Fusion for controlled cross-modal interaction.

Main Results:

• Achieved high classification accuracies of 85.76% on the challenging SSW60 dataset and 96.67% on the Birds21 dataset.
• Outperformed existing unimodal methods and mainstream fusion strategies, demonstrating superior performance.
• Analyses confirmed adaptive modality contribution adjustment based on discriminative evidence.

Conclusions:

• CABIF-Net offers an effective solution for fine-grained audio-visual bird species recognition, even with noisy and imbalanced data.
• The framework's adaptive fusion strategy enhances robustness and accuracy in ecological monitoring applications.