FF-ViT: probe orientation regression for robot-assisted endomicroscopy tissue scanning

Area of Science:

• Medical imaging
• Robotic surgery
• Artificial intelligence

Background:

• Probe-based confocal laser endomicroscopy (pCLE) is crucial for intraoperative cellular tissue visualization.
• Accurate probe positioning (contact and perpendicularity) is vital for high-quality pCLE imaging.
• Current robotic systems lack precise probe placement due to reliance on macroscopic vision.

Purpose of the Study:

• To develop an automated method for regressing pCLE probe orientation relative to the tissue surface.
• To enable precise probe positioning using endomicroscopic vision during robotic scanning.

Main Methods:

• A novel method utilizing the fast Fourier vision transformer (FF-ViT) for probe orientation regression.
• FF-ViT employs blur mapping attention (BMA) to refine representations.
• Integration with a pyramid angle regressor (PAR) for precise orientation estimation.

Main Results:

• Creation of the first pCLE probe-tissue orientation (pCLE-PTO) dataset.
• The proposed network achieves superior accuracy, stability, and generalizability compared to existing methods.
• The system demonstrates low computational complexity (1.8G FLOPs) and high inference speed (90 fps).

Conclusions:

• The developed framework holds significant clinical value for intraoperative tissue scanning.
• Potential for seamless integration into surgical robotic platforms.
• Enhances the precision and reliability of robotic-assisted pCLE procedures.