Scene-graph-driven semantic feature matching for monocular digestive endoscopy.

Area of Science:

• Medical Imaging
• Computer Vision
• Surgical Navigation

Background:

• Accurate 3D reconstruction of soft tissue surfaces is crucial for registering preoperative 3D models with endoscopic videos.
• Existing feature matching methods struggle with soft tissue deformation and smooth surfaces, limiting registration accuracy.

Purpose of the Study:

• To develop a new semantic description for enhanced feature matching in 3D reconstruction for endoscopy surgical navigation.
• To improve the accuracy and density of feature correspondence in challenging soft tissue environments.

Main Methods:

• A novel semantic feature descriptor integrating contour and SIFT features using a scene graph approach.
• A clustering algorithm designed based on the proposed semantic feature descriptor.
• Application of the semantic description within a structure from motion (SfM) reconstruction framework.

Main Results:

• Feature matching accuracy improved to 75.6%, with a 16.6% increase in pose estimation accuracy.
• SfM results showed a 39.8% increase in sparse points and 35.31% more valid points for 3D reconstruction training.
• Validation through phantom tests and real surgery videos, demonstrating superiority over typical methods.

Conclusions:

• The new semantic feature description enables more accurate and dense feature correspondence, providing crucial local semantic information.
• The novel approach is effective and robust for 3D reconstruction in clinical endoscopy applications.