Scaling up 3D Kernels with Bayesian Frequency Re-parameterization for Medical Image Segmentation.

Area of Science:

• Medical Image Analysis
• Computer Vision
• Deep Learning Architectures

Background:

• Large kernel (LK) sizes in Convolutional Neural Networks (CNNs) aim to increase the Effective Receptive Field (ERF) for medical image segmentation.
• However, performance saturation and degradation occur with excessively large kernels, hindering optimal locality learning.
• Structural Re-parameterization (SR) improves convergence with small kernels but can impact computational efficiency.

Purpose of the Study:

• To propose RepUX-Net, a pure CNN architecture utilizing a novel large kernel block design for improved medical image segmentation.
• To address the limitations of traditional large kernel convolutions and enhance convergence properties.
• To compete with and surpass existing state-of-the-art (SOTA) segmentation networks.

Main Methods:

• Developed RepUX-Net, a CNN architecture featuring a simple yet effective large kernel block.
• Derived an equivalency between kernel re-parameterization and kernel convergence variation.
• Introduced element-wise kernel convergence variation, inspired by spatial frequency, and modeled it as a Bayesian prior for weight re-parameterization during training.
• Utilized a reciprocal function to estimate frequency-weighted values for rescaling kernel elements in stochastic gradient descent.

Main Results:

• RepUX-Net demonstrated superior performance compared to 3D SOTA benchmarks across six challenging public datasets.
• Achieved consistent improvements in Dice Score for internal validation (FLARE), external validation (MSD, KiTS, LiTS, TCIA), and transfer learning (AMOS) scenarios.
• Consistently outperformed existing methods, showcasing the efficacy of the proposed large kernel block design and re-parameterization strategy.

Conclusions:

• RepUX-Net offers a competitive and efficient alternative for medical image segmentation, outperforming current SOTA methods.
• The proposed method effectively addresses the limitations of large kernel convolutions by incorporating spatial frequency-inspired re-parameterization.
• The architecture provides a strong foundation for future research in deep learning-based medical image analysis.