TinyUSFM: Towards Compact and Efficient Ultrasound Foundation Models.

Area of Science:

• Medical Imaging AI
• Foundation Models
• Ultrasound Diagnostics

Background:

• Foundation models excel in medical imaging but require extensive computational power.
• Deployment in resource-limited clinical settings is challenging due to high computational demands.

Purpose of the Study:

• To develop the first lightweight ultrasound foundation model (TinyUSFM) that retains the performance of large-scale models.
• To achieve computational efficiency without compromising organ versatility and task adaptability.

Main Methods:

• Knowledge distillation from a large Ultrasound Foundation Model (USFM) using curated small datasets.
• Feature-gradient driven coreset selection for high-quality, compact training data.
• Domain-separated masked image modeling and consistency-driven dynamic distillation for knowledge transfer.

Main Results:

• TinyUSFM achieves performance comparable to USFM using only 6.36% of parameters and 6.40% of GFLOPs.
• Outperforms vanilla models by 9.45% in classification and 7.72% in segmentation.
• Achieves 84.91% average classification accuracy and 85.78% average segmentation Dice score on the UniUS-Bench.
• Won first place in the MICCAI2025 IUGC Challenge.

Conclusions:

• TinyUSFM successfully bridges the gap between high-performance AI models and practical clinical deployment.
• The model offers significant computational efficiency, enabling wider accessibility of advanced ultrasound diagnostics.
• This work demonstrates the potential of lightweight foundation models for real-world medical applications.