PromptReg: Interactive Registration by "Corresponding Prompts" for Segment Anything Model (SAM).

Area of Science:

• Medical Imaging
• Computer Vision
• Image Processing

Background:

• Image registration is crucial for establishing pixel-level correspondence between images.
• Existing methods often rely on dense displacement fields (DDFs) or iterative transformations.
• Region-based representations offer an alternative by segmenting corresponding regions of interest (ROIs).

Purpose of the Study:

• To develop a novel, training-free, and non-iterative image registration algorithm.
• To leverage pre-trained segmentation models for identifying corresponding ROIs.
• To re-envision image registration as a paired ROI segmentation problem.

Main Methods:

• Utilized Segment Anything Model (SAM) for training-free ROI segmentation.
• Proposed a "corresponding prompt problem" to find paired ROIs across images.
• Developed an "inverse prompt" solution using Jacobian of prototypical features.
• Introduced a registration algorithm by marginalizing inverted prompts and spatial warping.

Main Results:

• Achieved competitive performance across diverse applications including medical (prostate, abdomen, lung MR/CT, histopathology) and non-medical (aerial) images.
• Demonstrated superior performance compared to intensity-based iterative and learning-based registration methods.
• Outperformed weakly-supervised registration methods that require fully-segmented training data.

Conclusions:

• The proposed training-free, non-iterative registration method is highly effective and versatile.
• Leveraging pre-trained models and a region-based approach offers significant advantages over traditional methods.
• This approach advances the field of image registration, particularly for applications demanding accuracy and efficiency.