Topology-aware segmentation for tubular structure in 3D microscopy

High-resolution 3D microscopy has become a foundational tool in biomedical research by reconstructing vascular and neural networks from subcellular to organ scales, thereby facilitating quantitative analysis of tissue microenvironments, disease progression, and early pathological changes. However, accurately segmenting tubular structure like microvascular and neurite in 3D microscopy remains challenging due to their densely packed, span a wide range of calibers, and branch extremely frequently. These characteristics hinder existing deep learning-based segmentation methods from simultaneously enforcing global connectivity and locally plausible radius profiles, often resulting in fragmented branches, spurious connections, and missing fine terminal processes. In this work, we propose a fully 3D topology-supervised segmentation framework for tubular structure that improves connectivity preservation and morphological consistency. Firstly, we introduce a radius-aware topology that integrates local radius estimates directly into connectivity constraints, so that the network simultaneously enforces correct continuity and physiologically credible thickness. Additionally, we implement adaptive topology-error modulation, which amplifies supervision in regions with significant topological deviations. This mechanism directs the network's focus toward critical errors (e.g. breaks and misjoins) rather than allowing sparse annotations or class imbalance to dominate optimization. Furthermore, we employ efficient large-receptive-field convolutions to capture long-range directional continuity in volumetric data, effectively recovering low-contrast, distal, thin-caliber terminal branches. We validate the approach on three publicly available modalities-electron microscopy vasculature, optical microscopy vasculature, and fluorescently labeled neuronal fibers-and observe that it preserves global 3D connectivity and avoids implausible radius jumps while maintaining strong voxel-level accuracy. Quantitatively, the proposed method achieves the highest Dice/clDice scores on SELMA3D (0.8839/0.9173), Mini-vessel (0.8842/0.9097), and FISBe (0.7803/0.8033), outperforming the strongest competing baselines across all three datasets. These results suggest consistent performance across different microscopy settings without requiring organ-specific anatomical priors.