Deep point cloud landmark localization for fringe projection profilometry.

Area of Science:

• Computer Vision
• Robotics
• Medical Imaging

Background:

• Point clouds offer richer information than images, making them valuable for advanced applications.
• Fringe Projection Profilometry (FPP) is a key technique for acquiring 3D point cloud data, with emerging use in robotic surgery vision systems.
• Current methods for robotic suturing landmarks using fluorescent fiducials increase system complexity and raise safety concerns.

Purpose of the Study:

• To develop a novel numerical landmark localization algorithm for semi-autonomous robotic suturing.
• To eliminate the need for fluorescent fiducials, thereby reducing system complexity and safety risks.
• To enhance the precision and reliability of suture landmark identification in surgical robotics.

Main Methods:

• A Convolutional Neural Network (CNN) is employed to generate landmark heatmaps from four-channel image data acquired via FPP.
• A Conditional Random Field (CRF) model is utilized to refine landmark coordinates by incorporating local and global shape constraints.
• The algorithm is designed to effectively tune landmark positions, reject extraneous landmarks, and recover any missed landmarks.

Main Results:

• The proposed CNN-CRF algorithm demonstrates robust performance in landmark localization.
• The method successfully refines landmark accuracy by leveraging shape constraints.
• Experimental validation on ex vivo porcine intestine tissue confirms the algorithm's effectiveness.

Conclusions:

• The developed numerical landmark localization algorithm offers a safer and less complex alternative to fluorescent fiducials for robotic surgery.
• The integration of CNNs and CRFs provides a powerful approach for precise landmark identification in 3D vision systems.
• This technique has the potential to significantly advance the capabilities of semi-autonomous robotic suturing systems.