Dual-modality endoscopic probe for tissue surface shape reconstruction and hyperspectral imaging enabled by deep...

Area of Science:

• Biomedical Engineering
• Medical Imaging
• Surgical Technology

Background:

• Accurate intra-operative data on tissue surface shape and spectral information is crucial for surgical guidance and decision-making.
• Existing endoscopic systems often lack the capability for simultaneous real-time 3D reconstruction and hyperspectral imaging.

Purpose of the Study:

• To develop and validate a dual-modality endoscopic system capable of real-time tissue surface shape reconstruction and hyperspectral imaging.
• To improve intra-operative data acquisition for enhanced surgical guidance.

Main Methods:

• A novel endoscopic probe with a unique fiber bundle arrangement and miniature optics was designed.
• Structured light projection with a Convolutional Neural Network (CNN) model enabled real-time 3D surface reconstruction.
• A CNN-based super-resolution model (SSRNet) was developed for hyperspectral imaging from sparse signals.

Main Results:

• The system achieved real-time 3D surface reconstruction at approximately 12 FPS and hyperspectral imaging at approximately 2 FPS.
• The 2.1 mm diameter probe is compatible with standard endoscope working channels.
• Validation on phantoms and ex vivo/in vivo tissues, with initial successful patient measurements during laryngeal surgery.

Conclusions:

• The dual-modality endoscopic system provides accurate, real-time shape and spectral data of tissue surfaces.
• The system's ability to capture data in a single snapshot minimizes the impact of tissue movement.
• The validated system shows significant potential for real-world clinical applications in surgery.