CDP-KDNet: Curriculum-Guided Dynamic Pruning and Knowledge Distillation for Resource-Efficient Ultrasound...

Area of Science:

• Medical imaging
• Machine learning
• Ultrasound technology

Background:

• Convolutional neural network (CNN)-based optical flow models excel in radio-frequency (RF) ultrasound and B-mode (BM) motion estimation.
• Complex CNN architectures present deployment challenges on resource-constrained devices due to high parameter counts and computational costs.

Purpose of the Study:

• To develop a novel, compressed deep learning model for ultrasound motion estimation that reduces complexity while maintaining performance.
• To integrate dynamic pruning, knowledge distillation, and curriculum learning for efficient model compression.

Main Methods:

• Developed a teacher network (UMEN-Net) for unsupervised motion estimation.
• Created a pruned sub-network (DP-Net) and applied compression techniques to yield the final model (CDP-KDNet).
• Evaluated CDP-KDNet on simulated, phantom, and in vivo ultrasound data.

Main Results:

• CDP-KDNet demonstrated superior Signal-to-Noise Ratio (SNR) and Contrast-to-Noise Ratio (CNR) for axial strain estimation compared to DP-Net and other lightweight CNNs.
• The model achieved performance comparable to the teacher network, using only 45.3% of parameters and 67.8% of floating-point operations.
• As an unsupervised model, CDP-KDNet eliminates the need for ground-truth labels during training.

Conclusions:

• CDP-KDNet offers a significant reduction in model complexity for ultrasound motion estimation.
• The model provides a promising, efficient, and label-free solution for deploying advanced motion estimation techniques on devices with limited resources.