SonoNERFs: Neural Radiance Fields Applied to Biological Echolocation Systems Allow 3D Scene Reconstruction through...

Area of Science:

• Bioacoustics
• Computational Neuroscience
• Computer Vision

Background:

• Bats use echolocation for navigation and hunting, interpreting complex acoustic signals.
• Optical Neural Radiance Fields (NeRFs) excel at 3D scene reconstruction from images.
• Interpreting echolocation data, especially without phase information, presents significant challenges.

Purpose of the Study:

• To adapt Neural Radiance Fields (NeRFs) for modeling bat echolocation.
• To develop a computational framework for understanding acoustic scene representation in echolocating animals.
• To reconstruct 3D environments from simulated bat echolocation data.

Main Methods:

• Introduced SonoNERF, a Neural Reflectivity Field model for acoustic environments.
• Leveraged principles from optical NeRFs and biological echolocation.
• Simulated echolocation data to train and test the SonoNERF model on various scenes.

Main Results:

• SonoNERF successfully reconstructs 3D scenes from echolocation data.
• The model predicts echo spectrograms for novel echolocation poses.
• Generated mesh-based and energy-based scene representations, demonstrating efficacy on complex scenarios including predator-prey interactions.

Conclusions:

• SonoNERF provides a novel framework for modeling acoustic scene understanding in bats.
• This work bridges the gap between computational vision and bioacoustics.
• The model offers a first-order approximation of how echolocating animals achieve scene perception.