Endo-TTAP: Robust Endoscopic Tissue Tracking via Multi-Facet Guided Attention and Hybrid Flow-point Supervision

Accurate tissue point tracking in endoscopic videos is crucial for robotic-assisted surgical navigation and scene understanding, yet remains challenging due to complex tissue deformations, instrument occlusions, and the scarcity of dense trajectory annotations. Existing methods struggle with long-term tracking robustness under these conditions because they primarily rely on local motion cues that fail in textureless regions and lack mechanisms to quantify tracking reliability. Without high-level semantic context to differentiate homogeneous tissues and explicit uncertainty modeling to prevent error accumulation during occlusions, existing trackers inevitably drift over time. We present Endo-TTAP, addressing these challenges through a two-stage hybrid supervision approach. Our method introduces: (1) A Multi-Facet Guided Attention (MFGA) module that fuses multi-scale optical flow features, semantic embeddings, and motion patterns via guided attention to jointly predict point positions, occlusion states, and tracking uncertainty; (2) An Auxiliary Curriculum Adapter (ACA) enabling progressive domain adaptation from synthetic to surgical data through exponential scheduling; (3) A Pseudo Label Generator (PLG) that creates high-quality dense annotations from sparse surgical data. Our two-stage training strategy first initializes components using synthetic datasets with optical flow ground truth, then transitions to real surgical data through unsupervised flow consistency and semi-supervised pseudo-label learning. We further contribute the Endo-TTAPC5 dataset, comprising 250 video segments across five clinically meaningful challenges. Extensive validation on two public datasets (SurgT, STIR) and our Endo-TTAPC5 dataset demonstrates that Endo-TTAP achieves state-of-the-art performance in tissue point tracking, particularly in complex endoscopic scenarios. Code and video demo are available at https://adampc888.github.io/Endo_TTAP/.