Real-Time Segmentation and Classification of Birdsong Syllables for Learning Experiments

Songbirds are essential for studying neuronal mechanisms of learned vocalizations. Closed-loop interventions require online recognition of a specific target syllable while the bird is singing, for example, for manipulation of auditory feedback, song-triggered neuronal microstimulation or optogenetics. Existing tools for closed-loop interventions can recognize only single syllables through manually created templates, with limited flexibility to adapt to new experiments. We here present Moove (Marking Online using only the Onsets of Vocal Elements), a novel neural network approach to real-time syllable segmentation and classification of Bengalese finch songs. Moove's two-stage architecture detects syllable onsets and offsets and classifies syllables using acoustic information only from the first part of the syllable, enabling precise temporal contingency between behavior and feedback. We verify Moove's fast and accurate online annotation of all recorded syllables in five adult male Bengalese finches (Lonchura striata domestica). To validate Moove as a tool for learning experiments, we trained one adult male Bengalese finch with an established protocol: a specific target syllable is covered with noise, which masks auditory feedback and leads the bird to introduce specific modifications to syllable sequencing. The trained bird learned to avoid the targeted syllable sequence with comparable outcomes to previous reinforcement learning experiments. Our results show that Moove can correctly segment and classify Bengalese finch syllables in real time, with speed and reliability that allows effective operant conditioning experiments. Moove could be used for other closed-loop experiments on vocal signals, making it a crucial tool for future investigations of birdsong sequencing.