Universal Scale Transformer for Histology Image Segmentation

Histology image segmentation is a critical prerequisite to pathological diagnosis. Accurate segmentation of these images can significantly aid physicians by facilitating quicker and more precise diagnostic decisions. A notable challenge in this area arises from the fact that different objects within histology images require segmentation at varying magnifications. However, most existing models are typically limited to performing segmentation at one pre-determined magnification. In this paper, we propose a novel universal scale transformer model (UniScaleFormer), which employs a scale-aware approach and integrates textual input to uniformly segment objects in histology images across various magnifications. Our method adopts an end-to-end architecture and utilizes a candidate mask query mechanism, specifically designed to identify and distinguish objects at different image scales. Moreover, we develop a Scale-Aware Module that enhances our network's ability to recognize the magnification level of input histology images, by using a scale query with extracted visual features. Then the scale query is integrated with mask queries, facilitating the incorporation of scale information. Experimental results demonstrate that the proposed method effectively achieves competitive results on various segmentation benchmarks at different magnifications. The code will be released at https://github.com/lhaof/UniScale.