SparseMed3D: Foundation Models for Sparse Instance Medical Segmentation

Vision foundation models excel at general segmentation but underperform sharply when positive instances occupy a vanishing fraction of the input-a regime that arises across small-lesion and rare-event medical segmentation, with neonatal hypoxic-ischemic encephalopathy (HIE) as a representative extreme case (lesions <1% of brain volume; positive-to-negative voxel ratio >99:1). We present SparseMed3D, a framework for adapting vision foundation models to this extreme-sparsity regime through three coupled components: (i) parameter-efficient multi-channel patch-embedding adaptation; (ii) diffusion-based multi-channel image fusion with a provable descent property; and (iii) patch-based inference with variance bounds and concentration inequalities under arbitrarily correlated patch predictions. The unified theoretical analysis characterizes aggregation variance through an effective sample size that quantifies the correlation penalty between overlapping patches, and yields an end-to-end error bound that couples fusion and aggregation errors. We instantiate the framework on the BONBID-HIE 2023 benchmark, where the empirical aggregation variance closely matches the theoretical scaling across stride configurations (std 0.024 vs. 0.233), and the resulting model doubles baseline SAM-Med3D Dice (0.24 $\to$ 0.48), recovering 77% of state-of-the-art specialized-pipeline performance without ensembles, task-specific architectural redesign, or extensive fine-tuning of the foundation backbone.