Time-of-flight fluorescence depth mapping using a spatiotemporal deep learning model

Area of Science:

• Medical Imaging
• Biophotonics
• Artificial Intelligence

Background:

• Fluorescence-guided surgery (FGS) uses molecular contrast agents for real-time visualization but is limited by optical artifacts like diffusion and absorption.
• These artifacts hinder accurate quantitative assessment of fluorophore distribution, potentially compromising surgical precision.
• Accurate sub-surface fluorescence imaging requires resolving topological depth maps at the millimeter scale.

Purpose of the Study:

• To develop a spatiotemporal deep learning architecture for rapid recovery of fluorophore depth topology using picosecond single-photon avalanche diode (SPAD) sensor images.
• To enable precise surgical decision-making by overcoming limitations in current FGS techniques.
• To achieve millimeter spatial resolution in wide-field, epi-illumination geometry for sub-surface fluorescence imaging.

Main Methods:

• A ConvLSTM-UNet deep learning network was developed for analyzing picosecond time-resolved images.
• The network was trained on simulated spatiotemporal maps using optical Monte Carlo methods and the instrument response function (IRF).
• The system utilized a SwissSPAD2 sensor synchronized with a picosecond laser diode, processing data from 10 temporal gates.

Main Results:

• The deep learning network successfully reconstructed fluorescence topography up to 15 mm depth.
• Reconstruction accuracy achieved a mean absolute error of less than 0.6 mm and mean depth variances below 0.5 mm.
• The system demonstrated a rapid inference time of approximately 30 ms.

Conclusions:

• Integrating temporal and spatial deep learning networks enables accurate depth mapping from time-resolved fluorescence data.
• The use of real IRF confirmed the applicability of SPAD sensors for sub-surface fluorescence mapping.
• This approach enhances the capabilities of fluorescence-guided surgery by providing quantitative depth information.