Real-Time Segmentation and Classification of Birdsong Syllables for Learning Experiments

Area of Science:

• Neuroscience
• Bioacoustics
• Machine Learning

Background:

• Songbirds are crucial models for understanding learned vocalizations.
• Closed-loop interventions require real-time syllable recognition for experiments like auditory feedback manipulation.
• Existing tools lack flexibility and adaptability for real-time song analysis.

Purpose of the Study:

• To introduce Moove (Marking Online using only the Onsets of Vocal Elements), a novel neural network for real-time birdsong syllable segmentation and classification.
• To enable precise temporal control for closed-loop experiments in songbirds.
• To validate Moove's effectiveness in operant conditioning paradigms.

Main Methods:

• Moove employs a two-stage neural network architecture for syllable onset/offset detection and classification.
• It utilizes acoustic information from the initial part of syllables for rapid analysis.
• The system was validated on five Bengalese finches and used in a reinforcement learning experiment.

Main Results:

• Moove achieved fast and accurate online annotation of all syllables in recorded Bengalese finch songs.
• A trained finch successfully modified its song sequence in response to masked auditory feedback, demonstrating Moove's utility in learning experiments.
• The system exhibited speed and reliability suitable for operant conditioning.

Conclusions:

• Moove provides a robust and adaptable tool for real-time birdsong analysis.
• Its capabilities facilitate advanced closed-loop experiments, including vocal learning studies.
• The open-source nature of Moove promotes its application across various vocal signal research.