TweetyBERT: Automated parsing of birdsong through self-supervised machine learning

Area of Science:

• Bioacoustics
• Computational Neuroscience
• Machine Learning

Background:

• Current analysis of animal vocalizations relies on human-labeled data, which is time-consuming and limits scalability.
• Unsupervised methods for parsing complex animal communication signals remain a significant challenge in bioacoustics.

Purpose of the Study:

• To introduce TweetyBERT, a novel self-supervised transformer neural network designed for the unsupervised analysis of birdsong.
• To demonstrate the capability of TweetyBERT to autonomously learn and identify units of communication in animal vocalizations.

Main Methods:

• Developed TweetyBERT, a transformer neural network utilizing a self-supervised learning approach (predicting masked audio fragments).
• Trained TweetyBERT on canary song data without human-provided labels or supervision.
• Applied the model to analyze acoustic and temporal patterns in canary vocalizations.

Main Results:

• TweetyBERT successfully learned to identify distinct behavioral units of birdsong, including notes, syllables, and phrases, from unlabeled audio data.
• The model captured intricate acoustic and temporal structures within the canary song autonomously.
• Demonstrated the potential for self-supervised learning in uncovering communication patterns in animal vocalizations.

Conclusions:

• Self-supervised models like TweetyBERT offer a powerful solution for analyzing large volumes of unlabeled animal vocalization data.
• This approach significantly accelerates the study of animal communication by removing the bottleneck of manual data annotation.
• TweetyBERT represents a significant advancement in computational bioacoustics and the automated analysis of animal sounds.