Fine-Tuning U-Net for Ultrasound Image Segmentation: Different Layers, Different Outcomes

Area of Science:

• Medical Imaging
• Deep Learning
• Computer Vision

Background:

• Scarce and expensive data in medical deep learning necessitates transfer learning.
• Traditional transfer learning often keeps shallow layers fixed, modifying deeper ones.
• This conventional approach may be suboptimal for U-Net architectures and domain shifts, like to ultrasound (US) imaging.

Purpose of the Study:

• To investigate the impact of fine-tuning different layer sets in a pretrained U-Net for US image segmentation.
• To compare strategies for layer selection in transfer learning for medical image segmentation tasks.
• To determine optimal fine-tuning approaches for U-Net models applied to US images.

Main Methods:

• Investigated fine-tuning of various layer combinations in a pretrained U-Net model.
• Analyzed two distinct schemes defining shallow and deep layers.
• Evaluated performance on simulated US images, two human US datasets, and a chest X-ray dataset.

Main Results:

• The choice of layers to fine-tune significantly impacts segmentation performance.
• Fine-tuning only the deepest layers, a common practice for classification, proved to be the least effective strategy.
• Fine-tuning shallow layers yielded superior results for US image segmentation with U-Net models.

Conclusions:

• Fine-tuning shallow layers is a more appropriate strategy than deep layers for US image segmentation using U-Net.
• Shallow layers capture essential low-level features critical for medical image segmentation.
• Fine-tuning shallow layers offers a faster and more effective approach compared to full network fine-tuning, even with substantial US data.