Machine Learning-Based View Synthesis in Fourier Lightfield Microscopy

Area of Science:

• Microscopy
• Computational Imaging
• Optics

Background:

• Fourier lightfield microscopy (FLFM) enables 3D imaging of dynamic samples.
• FLFM captures multiple views in one shot but faces a trade-off between view number and lateral resolution.
• Generating more views without resolution loss is crucial for FLFM applications.

Purpose of the Study:

• To investigate the application of neural radiance fields (NeRF) for synthesizing high-resolution views in FLFM.
• To evaluate the effectiveness of NeRF in improving lateral resolution and parallax in FLFM.

Main Methods:

• Adaptation of a NeRF-based view synthesis method, initially developed for macroscopic scenes.
• Application of the NeRF method to images acquired using Fourier lightfield microscopy.
• Analysis of synthesized views based on lateral resolution and parallax.

Main Results:

• The NeRF method successfully synthesized a higher number of views for FLFM.
• The synthesized views maintained high lateral resolution.
• Continuous and realistic parallax was achieved in the synthesized views.
• The technique proved efficient in epi-illumination microscopy mode.

Conclusions:

• Neural radiance fields offer an effective solution for view synthesis in Fourier lightfield microscopy.
• This approach overcomes the inherent view-number/resolution trade-off in FLFM.
• The method enhances the quality and utility of 3D imaging with FLFM, particularly in epi-illumination setups.