Fast Multi-Organ Fine Segmentation in CT Images with Hierarchical Sparse Sampling and Residual Transformer

Multi-Organ segmentation of 3D medical images is fundamental with meaningful applications in various clinical automation pipelines. Although deep learning has achieved superior performance, the time and memory consumption of segmenting the entire 3D volume voxel by voxel using neural networks can be huge. Classifiers have been developed as an alternative in cases with certain points of interest, but the trade-off between speed and accuracy remains an issue. Thus, we propose a novel fast multi-organ segmentation framework with the usage of hierarchical sparse sampling and a Residual Transformer. Compared with whole-volume analysis, the hierarchical sparse sampling strategy could successfully reduce computation time while preserving a meaningful hierarchical context utilizing multiple resolution levels. The architecture of the Residual Transformer segmentation network could extract and combine information from different levels of information in the sparse descriptor while maintaining a low computational cost. In an internal data set containing 10,253 CT images and the public dataset TotalSegmentator, the proposed method successfully improved qualitative and quantitative segmentation performance compared to the current fast organ classifier, with fast speed at the level of ∼2.24 seconds on CPU hardware. The potential of achieving real-time fine organ segmentation is suggested.Clinical relevance- We introduce an innovative fast multi-organ segmentation framework that utilizes hierarchical sparse sampling combined with a Residual Transformer. This approach significantly reduces computation time compared to whole-volume analysis while retaining meaningful hierarchical context through multiple resolution levels. This method enhances both qualitative and quantitative segmentation performance over existing fast organ classifiers, achieving segmentation in approximately 2.24 seconds on standard CPU hardware. This indicates the promising potential for real-time fine organ segmentation in various clinical applications, including scan registration, lesion detection, and landmarking.