AVB81: A Fine-Grained Audiovisual Dataset and Benchmark for Bird Classification in the Wild

Automated bird monitoring plays a crucial role in ecoinformatics and biodiversity conservation. Despite significant advancements in fine-grained visual classification, targets in complex wild environments frequently encounter interferences such as long capture distances, severe visual occlusion, and strong background noise, rendering single-modal perception highly susceptible to performance bottlenecks. To advance research in audiovisual multimodal bird classification, this paper introduces AVB81, a multimodal dataset tailored for fine-grained bird recognition. Covering 81 bird species in North America, the dataset comprises 3247 fixed-length 10 s field videos, meticulously supplemented with 5418 independent audios and 7083 high-quality static images. Based on this dataset, we systematically conduct single-modal performance evaluations and cross-paradigm audiovisual fusion experiments, establishing a comprehensive and in-depth multimodal evaluation benchmark. The experimental results demonstrate that in video classification tasks, audiovisual multimodal fusion methods significantly outperform single-modal baselines. Notably, the mid-fusion strategy based on deep semantic spaces achieves the optimal performance. Furthermore, the experiments objectively reveal the formidable challenges associated with audio recognition within videos captured in complex wild habitats. Overall, AVB81 exhibits exceptional adaptability for multimodal research, providing a highly challenging evaluation benchmark for multimodal semantic modeling in complex scenarios and laying a solid data foundation for the development and validation of next-generation intelligent ecological monitoring systems.