A deep learning approach for the analysis of birdsong

Deep learning tools for behavior analysis have enabled important insights and discoveries in neuroscience. Yet, they often compromise interpretability and generalizability for performance, making quantitative comparisons across datasets difficult. We developed a novel deep learning-based behavior analysis pipeline, Avian Vocalization Network (AVN), for zebra finch learned vocalizations - the most widely studied vocal learning model species. AVN annotates songs with high accuracy, generalizing across multiple animal colonies without re-training, and generates a comprehensive set of interpretable features describing song syntax, timing, and acoustic properties. We use this feature set to compare song phenotypes across research groups and experiments and to predict a bird's stage in song development. Additionally, we have developed a novel method to measure song imitation that requires no training data for new comparisons and outperforms existing similarity scoring methods in its sensitivity and agreement with expert human judgements. These tools are available through the open-source AVN python package and graphical application, making them accessible to researchers without prior coding experience. Altogether, this behavior analysis toolkit stands to accelerate the study of vocal behavior by standardizing phenotype and learning outcome mapping, thus helping scientists better link behavior to the underlying neural processes.