Optimization in Sparse 2D to Dense 3D Weakly Supervised Learning: Application to Multi-Label Segmentation of Large ex...

The fully supervised segmentation of 3D medical images featuring large amounts of slices (1,000+), such as serial block-face electron microscopy, or high-resolution ex vivo MRI, imposes prohibitive cost of volumetric manual annotation. Oftentimes, researchers resort to limited ground truth annotations consisting of sparse 2D slices, forcing a reliance on models that lack volumetric context. Incorporating across-slice information in the model can be done using a weakly supervised Sparse-to-Dense framework. However, guidelines for training such models remain ambiguous, specifically regarding the alignment between human-centric visual enhancements and machine perception, and the transferability of optimization strategies from 2D to 3D contexts. Here, we propose an analysis of divergent regularization needs in the context of multi-class segmentation of high-resolution ex vivo spinal cord MRI.

Data consists of 9.4T MRI of ex vivo spinal cord volumes with sparse manual annotations (428 annotated slices). Samples were obtained from pathologically confirmed cases of Multiple Sclerosis, totaling 104,000+ slices. A 2D Teacher network was trained on those sparse annotations to generate dense volumetric pseudo-labels, which were subsequently used to train a 3D Student network. We conducted a systematic optimization of this pipeline, analyzing the impact of image preprocessing (to evaluate human-centric image contrast enhancement), spatial augmentation (to mitigate positional shortcut learning), and soft-label regularization (to model lesion boundary uncertainty) on both 2D and 3D architectures.

Our analysis reveals a critical divergence in training dynamics between dimensions that challenges standard low-data training assumptions. While the 2D Teacher significantly benefited from strong spatial augmentation and soft-labeling regularization - improving White Matter Lesion detection by over 11 Dice Score percentage points - these same techniques failed to improve or even degraded the performance of the 3D Student. Additionally, preprocessing techniques designed to enhance visual contrast for human raters (such as CLAHE) caused a performance drop, degrading Gray Matter Lesion Dice Scores by nearly 25 percentage points by disrupting global statistical cues.

This study sheds light on a perception divergence: preprocessing methods designed to aid human raters (image contrast enhancement) proved destructive to machine learning models by distorting global intensity statistics. Moreover, this study reveals a regularization conflict across dimensions in sparse-to-dense learning. While aggressive spatial augmentation and soft-labeling were helpful for the 2D Teacher to overcome sparse data scarcity, propagating these distinct techniques to the 3D Student resulted in performance degradation. These findings suggest that 3D architectures, when trained on dense pseudo-labels, exhibit fundamentally different optimization landscapes than their 2D counterparts and require a distinct, conservative regularization strategy. Code and model are available at https://github.com/ivadomed/model_seg_sc-gm-lesion_human_ms_exvivo_t2star.