AVB81: A Fine-Grained Audiovisual Dataset and Benchmark for Bird Classification in the Wild

Area of Science:

• Ecoinformatics
• Biodiversity Conservation
• Machine Learning

Background:

• Automated bird monitoring is vital for conservation, but visual methods struggle with wild environment challenges like occlusion and noise.
• Existing fine-grained visual classification techniques face performance bottlenecks in complex natural settings.
• Audiovisual multimodal approaches offer a promising avenue to overcome single-modal limitations in bird identification.

Purpose of the Study:

• Introduce AVB81, a novel multimodal dataset for fine-grained bird recognition.
• Establish a comprehensive benchmark for evaluating audiovisual fusion methods in ecological contexts.
• Advance the development of intelligent ecological monitoring systems.

Main Methods:

• Developed AVB81 dataset: 3247 videos, 5418 audios, 7083 images of 81 North American bird species.
• Conducted single-modal performance evaluations for visual and audio classification.
• Performed cross-paradigm audiovisual fusion experiments, including deep semantic space mid-fusion.

Main Results:

• Audiovisual multimodal fusion significantly outperformed single-modal baselines in video classification tasks.
• The mid-fusion strategy in deep semantic spaces yielded optimal performance.
• Audio recognition in complex wild habitats presents significant challenges, as evidenced by experimental results.

Conclusions:

• AVB81 provides a robust benchmark for multimodal semantic modeling in challenging ecological scenarios.
• Audiovisual fusion is superior to single-modal approaches for automated bird monitoring.
• The dataset supports the development of next-generation intelligent ecological monitoring systems.