Related Experiment Videos
Map2VidSeg Revisited: A controlled-upgrade study of classifier-derived prompts for medical video segmentation
Hanna Borgli1, Michael Riegler2, Håkon Kvale Stensland1
1Simula Research Laboratory, Kristian Augusts gate 23, Oslo, 0164, Norway; Department of Informatics, University of Oslo, P.O. Box 1080, Blindern, Oslo, 0316, Norway.
Abstract:
Map2VidSeg uses classifier-derived localization maps to generate prompts for video segmentation without segmentation training data. We revisit the pipeline under a fixed 224 × 224 visual-prompt regime and test whether newer foundation-model components improve performance. Starting from the CBMS 2025 system of DINOv2 attention, YOLOE, and SAM-2 on SUN-SEG polyp video, we replace the components along three axes: DINOv3 for map generation, YOLOE-26, for prompt-based segmentation, and SAM-3 for temporal refinement. A completed 2 × 2 prompt-source × YOLOE-stage factorial with SAM-3 fixed shows that the full DINOv3 + YOLOE-26 + SAM-3 upgrade reaches 0.6040 IoU and 0.6684 Dice on SUN-SEG Unseen, whereas the DINOv2 + YOLOE-11 + SAM-3 control reaches 0.6750 IoU and 0.7426 Dice. The prompt-source swap is near neutral with YOLOE-11, whereas YOLOE-26 is weaker at this prompt resolution and the full upgrade underperforms the additive expectation. In a matched comparison using identical cached YOLOE masks, SAM-3 tracker double-pass produced a modest paired gain over SAM-2 double-pass (+0.0275 IoU; 95% BCa CI [+0.0122,+0.0458]; p=0.0012). A second-task diagnostic study on CholecSeg8k binary instruments shows the same pattern: integration changes improve the YOLOE stage, whereas SAM-3 reduced performance on multi-tool clips under shared-identity tracking, and class-aware tracking eliminated this loss. The results support pipeline-level evaluation and identify class factoring as a third design axis alongside prompt quality and prompt-model compatibility.