Selecting the best optimizers for deep learning-based medical image segmentation.

Area of Science:

• Deep learning
• Medical image analysis
• Optimization algorithms

Background:

• Deep learning models for medical image segmentation often utilize stochastic gradient descent (SGD) variants.
• Adaptive learning and accelerated schemes, including momentum optimizers, are common training strategies.
• Understanding the interplay between learning rate (LR) and momentum rate (MR) is crucial for optimizing network performance.

Purpose of the Study:

• To explore optimal deep learning optimizers for medical image segmentation.
• To guide the design of segmentation networks with effective optimization strategies.
• To investigate the combined effect of adaptive and accelerated optimization methods.

Main Methods:

• Proposed a novel cyclic optimization method integrating cyclic learning and momentum rate (MR).
• The method is based on the Nesterov accelerated gradient optimizer.
• Evaluated the optimizer on cardiac image segmentation from MRI and CT scans using the ACDC challenge dataset and four network architectures.

Main Results:

• The proposed cyclic optimizer achieved over a 2% improvement in the dice metric compared to existing optimizers.
• Demonstrated superior performance in both single and multi-object segmentation settings.
• Achieved better results with similar or lower computational costs.

Conclusions:

• The combination of accelerated and adaptive optimization significantly impacts medical image segmentation performance.
• The proposed Cyclic Learning/Momentum Rate strategy improves efficiency and accuracy in deep learning-based medical image segmentation.
• The novel strategy offers better generalization compared to adaptive optimizers.