Temporal focusing multiphoton microscopy with cross-modality multi-stage 3D U-Net for fast and clear bioimaging

Area of Science:

• Biomedical optics
• Microscopy
• Computational imaging

Background:

• Temporal focusing multiphoton excitation microscopy (TFMPEM) offers fast widefield biotissue imaging with optical sectioning.
• Scattering effects in widefield illumination severely degrade imaging performance, especially in deep tissue layers, causing signal crosstalk and low signal-to-noise ratios.

Purpose of the Study:

• To develop a cross-modality learning-based neural network for image registration and restoration in TFMPEM.
• To improve the quality of TFMPEM images, particularly for deep-layer imaging, by mitigating scattering-induced artifacts.

Main Methods:

• An unsupervised U-Net model combined with VoxelMorph registration was used to align point-scanning multiphoton excitation microscopy images with TFMPEM images.
• A multi-stage 3D U-Net with cross-stage feature fusion and self-supervised attention was employed for inferring in-vitro TFMPEM volumetric images.
• Transfer learning was utilized by pretraining a 3D U-Net on in-vitro data and fine-tuning it with a small in-vivo dataset.

Main Results:

• The proposed method significantly improved Structure Similarity Index Measures (SSIMs) for in-vitro TFMPEM images (10-ms exposure) from 0.38 to 0.93 (shallow) and 0.80 (deep).
• Transfer learning enhanced in-vivo TFMPEM images (1-ms exposure), achieving SSIMs of 0.97 (shallow) and 0.94 (deep).

Conclusions:

• The cross-modality learning-based neural network effectively registers and restores TFMPEM images, overcoming scattering limitations.
• The developed method demonstrates robust performance for both in-vitro and in-vivo deep tissue imaging, paving the way for advanced biological visualization.