Stretcher: a learning-based framework for deformation-robust keypoint descriptors

Keypoint-based tracking and navigation are central to many image-guided surgical systems, yet their reliability degrades significantly in the presence of soft-tissue deformation, as commonly observed in procedures such as liver surgery. Existing keypoint descriptors often fail to remain stable under large affine and non-rigid transformations. This work introduces Stretcher, a framework designed to improve the deformation robustness of keypoint descriptors for surgical tracking and navigation.

Instead of directly matching keypoint descriptors across deformed images, Stretcher learns a neural model that explicitly captures the effect of affine deformations on descriptor representations. At inference time, a grid of affine transformations is simulated, and keypoint descriptors are efficiently adjusted using the learned model, avoiding redundant descriptor recomputation. This enables robust descriptor adaptation under deformation while maintaining computational efficiency.

Stretcher is evaluated on liver surgery images exhibiting large tissue deformation. Experimental results show that the proposed approach improves keypoint matching robustness in highly deformed regions. Across all evaluated scenarios, Stretcher maintains state-of-the-art precision while providing higher matching accuracy compared to existing descriptor-based methods, particularly where deformation is severe.

By explicitly modeling the impact of affine deformations on keypoint descriptors, Stretcher enables reliable keypoint matching under complex tissue motion. The proposed framework improves robustness in challenging surgical scenarios without sacrificing efficiency, making it well suited for deformation-prone intraoperative tracking and navigation tasks. Code is available at https://github.com/constiwitzleben/Stretchers .