Gradient-Based Self-supervised Multi-task Optimization for Weakly Supervised Brain Tumor Segmentation

Accurate brain tumor segmentation requires extensive voxel-wise annotations, making fully supervised training costly and time-consuming. Weak supervision alleviates this burden but often suffers from limited accuracy. To improve label efficiency, we propose DPS-SSL: Differentiable Policy Selection Self-Supervised Learning, a gradient-based, differentiable policy selection framework for weakly supervised 3D tumor segmentation within a self-supervised multi-task learning setting. Instead of relying on fixed pretext tasks, our method learns (end-to-end) which combinations of auxiliary self-supervised tasks (rotation, flip, noise injection, blurring, jigsaw, and patch masking) and their magnitudes most benefit the segmentation objective. A gradient-driven policy mechanism dynamically identifies relevant tasks and suppresses harmful ones, while adaptive loss weighting balances contributions between main and auxiliary tasks. Experiments on the BraTS2020 and BraTS2023 datasets show that our approach achieves Whole Tumor Dice scores of 90.7% and 90.3%, respectively, approaching fully supervised performance while requiring only weak annotations. These results demonstrate that task-adaptive self-supervision provides an effective and label-efficient solution for 3D brain tumor segmentation.