EndoMatcher: Generalizable Endoscopic Image Matcher via Multi-Domain Pre-training for Robot-Assisted Surgery

Area of Science:

• Computer Vision
• Medical Robotics
• Surgical Technology

Background:

• Dense feature matching in endoscopic images is vital for robot-assisted surgery but hindered by poor visual conditions and limited annotated data.
• Existing methods struggle with generalization across diverse endoscopic imaging scenarios.

Purpose of the Study:

• To develop a generalizable endoscopic image matcher, EndoMatcher, capable of robust dense feature matching.
• To address data scarcity and domain diversity challenges in endoscopic matching.

Main Methods:

• Proposed EndoMatcher, a two-branch Vision Transformer with dual interaction blocks for multi-scale feature extraction and robust correspondence learning.
• Introduced Endo-Mix6, a large-scale (1.2M pairs), multi-domain dataset for endoscopic matching, featuring real and synthetic data across six domains.
• Implemented a progressive multi-objective training strategy to handle dataset heterogeneity and ensure stable optimization.

Main Results:

• EndoMatcher significantly improved inlier matches by 140.69% and 201.43% on benchmark datasets.
• Achieved a 10% increase in Matching Direction Prediction Accuracy (MDPA) on the Gastro-Matching dataset, reaching 85.4%.
• Demonstrated zero-shot generalization capabilities to unseen organs and imaging conditions.

Conclusions:

• EndoMatcher provides accurate and generalizable dense feature matching for challenging endoscopic conditions.
• The developed multi-domain dataset and training strategy advance the field of endoscopic image analysis.
• The publicly available code facilitates further research and development in medical robotics and computer-assisted surgery.