VerTE-MT: A Multi-Task Framework with Entropy-Guided Sampling for Vertebrae Segmentation and Localisation in CT

Automated analysis of spinal CT involves localising, identifying and segmenting individual vertebrae. Challenges due to similarity of vertebral structures, pathological variations and rare cases (e.g. transitional vertebrae), lead to misclassifications. Prior approaches typically address segmentation and localisation separately, with multi-stage pipelines, disregarding their intrinsic relationship. This paper introduces VerTE-MT, a novel singlestage, multi-task (MT) learning framework that concurrently performs vertebrae segmentation and centroid localisation. The proposed architecture integrates a shared volumetric encoder, a Vision Transformer bottleneck for global spatial reasoning, and dual decoders for segmentation and localisation. Entropy-guided sampling dynamically prioritises under-represented vertebrae (e.g L6) enabling efficient MT learning and enhancing performance on pathological anatomies (e.g. scoliotic). In the VerSe'20 public and hidden test set, VerTE-MT outperforms existing singlestage methods, achieving average Dice score of 84.18% and 85.45% for vertebral column segmentation, 81.03% and 75.96% in L6, while reducing segmentation boundary errors with a decrease in Hausdorff Distance (HD) of up to 4.62mm and 4.04mm, respectively. It also obtains robust localisation maintaining a mean error below 10mm across spinal regions. Zero-shot validation on cadaveric and clinical CT scoliotic datasets, with mean Dice of 83.03% and 65.17% respectively, highlights VerTE-MT's potential on unseen pathological cases.