Physics-driven self-supervised learning for fast high-resolution robust 3D reconstruction of light-field microscopy

Light-field microscopy (LFM) and its variants have significantly advanced intravital high-speed 3D imaging. However, their practical applications remain limited due to trade-offs among processing speed, fidelity, and generalization in existing reconstruction methods. Here we propose a physics-driven self-supervised reconstruction network (SeReNet) for unscanned LFM and scanning LFM (sLFM) to achieve near-diffraction-limited resolution at millisecond-level processing speed. SeReNet leverages 4D information priors to not only achieve better generalization than existing deep-learning methods, especially under challenging conditions such as strong noise, optical aberration, and sample motion, but also improve processing speed by 700 times over iterative tomography. Axial performance can be further enhanced via fine-tuning as an optional add-on with compromised generalization. We demonstrate these advantages by imaging living cells, zebrafish embryos and larvae, Caenorhabditis elegans, and mice. Equipped with SeReNet, sLFM now enables continuous day-long high-speed 3D subcellular imaging with over 300,000 volumes of large-scale intercellular dynamics, such as immune responses and neural activities, leading to widespread practical biological applications.