Parallelized computational 3D video microscopy of freely moving organisms at multiple gigapixels per second

Area of Science:

• Biophysics
• Microscopy
• Computational Imaging

Background:

• Studying freely moving organisms requires high-speed 3D imaging.
• Current microscopy techniques compromise speed or resolution for 3D data acquisition.
• Simultaneously optimizing field of view, speed, and 3D resolution is a significant challenge.

Purpose of the Study:

• To present 3D-RAPID, a novel computational microscope for high-speed 3D topographic imaging.
• To overcome the limitations of existing techniques in capturing dynamic biological processes.
• To enable the study of collective behaviors in model organisms with high fidelity.

Main Methods:

• Utilized a synchronized array of 54 cameras for simultaneous image capture.
• Developed a self-supervised 3D reconstruction algorithm leveraging stereo overlap and ray propagation.
• Integrated a neural network for mapping photometric images to 3D topography.

Main Results:

• Achieved high-speed 3D topographic video capture over a 135 cm² area.
• Reached frame rates of up to 230 frames per second with spatiotemporal throughputs exceeding 5 gigapixels per second.
• Demonstrated robust 3D reconstruction for extended video sequences and scalable camera arrays.

Conclusions:

• 3D-RAPID successfully captures high-speed 3D behavior of freely moving organisms like ants, fruit flies, and zebrafish larvae.
• The system offers a significant advancement in non-invasive, high-throughput biological observation.
• This technology opens new avenues for understanding complex behaviors in ecological and developmental contexts.