EndoMatcher: Generalizable Endoscopic Image Matcher via Multi-Domain Pre-training for Robot-Assisted Surgery

Generalizable dense feature matching in endoscopic images is crucial for robot-assisted tasks, including 3D reconstruction, navigation, and surgical scene understanding. Yet, it remains challenging due to difficult visual conditions (e.g., weak textures, large viewpoint variations) and scarce annotated data. To address these challenges, we propose EndoMatcher, a generalizable endoscopic image matcher via large-scale, multi-domain data pre-training. EndoMatcher employs a two-branch Vision Transformer to extract multi-scale features, enhanced by dual interaction blocks for robust correspondence learning under difficult visual conditions. To overcome data scarcity and improve domain diversity, we construct Endo-Mix6, the first multi-domain dataset for endoscopic matching. Endo- Mix6 consists of approximately 1.2M real and synthetic image pairs across six domains, each corresponding to a distinct endoscopic data source or imaging scenario. Correspondence labels are generated through Structure-from-Motion and simulated transformations. The scale and cross-domain heterogeneity of Endo-Mix6 introduce new training challenges due to significant variations in dataset sizes, distribution shifts, and label noise. To ensure stable optimization, we adopt a progressive multi-objective training strategy that adaptively balances supervision across domains. This enhances feature learning and enables generalization to unseen organs and imaging conditions in a zero-shot fashion. Extensive zero-shot matching experiments demonstrate that EndoMatcher increases the number of inlier matches by 140.69% and 201.43% on the Hamlyn Centre and Bladder Tissue datasets over state-of-the-art methods, respectively, and improves the Matching Direction Prediction Accuracy (MDPA) from 76.0% to 85.4% on the Gastro-Matching dataset, achieving dense and accurate matching under challenging endoscopic conditions. The code is publicly available at https://github.com/Beryl2000/EndoMatcher.