Rapid robust high-fidelity 3D neuronal extraction from multiview calcium imaging datasets

Recent developments in imaging facilitate large-scale three-dimensional (3D) neuronal recording. While the resulting large datasets shed light on population-level neural coding, extracting neuronal calcium dynamics from 3D volumes remains more challenging than from two-dimensional images due to noise and scattering. Here we present DeepWonder3D, a general end-to-end pipeline for rapid and robust 3D neuronal extraction with high fidelity. Instead of processing voxel by voxel, DeepWonder3D works on the multiview projections of 3D imaging data obtained either digitally or optically through specific point spread functions and is therefore applicable to diverse techniques, including point-scanning microscopy, light-field microscopy and two-photon synthetic aperture microscopy. Integrating denoising, resolution registration, background removal, neuronal extraction and multiview fusion into a unified pipeline tailored for large-scale high-resolution datasets contaminated by noise and scattering, DeepWonder3D outperforms state-of-the-art methods in 3D localization accuracy with a tenfold reduction in computational costs, validated by numerical simulations and a hybrid two-photon/light-field imaging system. With the RUSH3D mesoscope, DeepWonder3D achieves high-fidelity 3D calcium extraction of tens of thousands of neurons across the mouse cortex within hours.