Learning spectro-temporal representations of complex sounds with parameterized neural networks.

Area of Science:

• Auditory Neuroscience
• Deep Learning
• Computational Neuroscience

Background:

• Deep learning models show promise in auditory neuroscience but often lack interpretability.
• Understanding the internal computations of these models is crucial for scientific advancement.

Purpose of the Study:

• To introduce a novel, interpretable neural network layer based on Gabor filters (learnable spectro-temporal filters - STRFs).
• To evaluate the performance and interpretability of STRF-based models across diverse auditory tasks.

Main Methods:

• Developed a parametrized neural network layer utilizing learnable spectro-temporal filters (STRFs).
• Evaluated STRF models on speech activity detection, speaker verification, urban sound classification, and zebra finch call classification.
• Analyzed learned spectro-temporal modulations using quantitative measures.

Main Results:

• STRF-based models achieved performance on par with state-of-the-art across all evaluated tasks.
• STRF models excelled in speech activity detection.
• Learned filters adapted to tasks, focusing on low temporal and spectral modulations, mirroring human auditory cortex findings.
• Task organization revealed clustering of human vocalizations separately from bird vocalizations and urban sounds.

Conclusions:

• The proposed interpretable STRF layer is effective for auditory neuroscience research.
• STRF models provide valuable insights into auditory processing mechanisms.
• The findings suggest a biological plausibility for learned spectro-temporal processing in auditory systems.