SparseMed3D: Foundation Models for Sparse Instance Medical Segmentation.

Area of Science:

• Medical Imaging
• Computer Vision
• Machine Learning

Background:

• Vision foundation models struggle with extreme class imbalance in medical segmentation.
• Conditions like neonatal hypoxic-ischemic encephalopathy (HIE) present challenges with lesions <1% of brain volume.

Purpose of the Study:

• To develop a framework, SparseMed3D, for adapting vision foundation models to extreme sparsity medical segmentation.
• To address the underperformance of general models in detecting rare and small medical instances.

Main Methods:

• Introduced SparseMed3D with three components: parameter-efficient patch-embedding adaptation, diffusion-based image fusion, and patch-based inference with variance bounds.
• Developed a unified theoretical analysis for aggregation variance and end-to-end error bounds.

Main Results:

• SparseMed3D demonstrated empirical aggregation variance matching theoretical scaling on the BONBID-HIE benchmark.
• The framework doubled the Dice score (0.24 to 0.48) compared to baseline SAM-Med3D.
• Achieved 77% of state-of-the-art performance without ensembles or extensive fine-tuning.

Conclusions:

• SparseMed3D effectively adapts vision foundation models for extreme sparsity medical segmentation.
• The framework offers a promising approach for rare-event and small-lesion detection in medical imaging.
• This method provides significant performance gains without requiring task-specific architectural changes or extensive retraining.