ScanSeg: Policy-Guided Sequential Prompting for Dense Fine-Grained Ultrasound Segmentation

Dense fine-grained segmentation on standardized ultrasound planes is clinically essential but remains challenging because multiple anatomical structures coexist under speckle noise, acoustic shadowing, weak boundaries, and partial visibility. Promptable foundation models can produce high-quality masks when queried, but they typically return one structure at a time and lack a mechanism to automatically decide what to segment next and where additional evidence is needed, making single-shot "auto" outputs brittle on complex planes. This paper introduces ScanSeg, a scan-inspired framework that formulates multi-structure ultrasound segmentation as sequential evidence accumulation via a small number of prompted calls. ScanSeg maintains a global per-pixel posterior over classes and iteratively issues prompts to a SAM predictor to obtain structure-specific mask observations with confidence scores. Each observation is fused into the posterior through a stable logit-space evidence accumulation rule, preserving global competition among structures. To enable efficient scanning, a lightweight proposal policy is learned to select the next target structure and prompt location conditioned on the current posterior and its uncertainty, trained by supervised imitation from step-wise oracle actions defined by posterior residuals. Experiments on dense abdominal and fetal cardiac ultrasound planes demonstrate that ScanSeg consistently outperforms both single-shot and prompted baselines, yielding more reliable fine-grained multi-structure segmentation and improving average Dice by up to 6.52 points on Abdomen and 4.94 points on A4C with only a few predictor calls.