VGFA: Variation-Robust Graph-Level Feature Alignment for Domain Adaptive Nuclei Detection

Accurate detection of nuclei is a crucial step in advancing pathology image analysis for disease diagnosis and treatment. However, significant domain discrepancies exist among pathology images, which severely degrade the performance of detection models. Despite promising results from existing domain adaptation approaches, they may overlook the detrimental impact of intra-domain variation (IDV) at both cell and image scales. The IDV issue typically manifests as dramatic differences in nuclei composition, morphology, and spatial arrangement, which occurs not only across different pathology images but also within a single image. This inherent heterogeneity produces highly complex feature distributions, ultimately making cross-domain alignment significantly more arduous. Moreover, the presence of IDV further injects noise into pseudo-labels, reducing the signal-to-noise ratio in the feature space and complicating the alignment process. To tackle these challenges, we propose a novel variation-robust graph-level feature alignment (VGFA) framework for unsupervised domain adaptive nuclei detection. Specifically, our method first incorporates a prior-based nuclei graph pruning scheme that harnesses nuclei spatial contextual priors and dynamically eliminates unreliable nodes from the nuclei graph. Then, a local-global nuclei encoding network is designed to learn nuclei graph representations that holistically encapsulate the consistent traits among various nuclei, thereby mitigating challenges posed by cell-scale IDV. Moreover, VGFA leverages a nuclei graph discrepancy loss that is resilient to image-scale IDV, achieving effective feature alignment in cross-domain graph feature space. Extensive experiments across different adaptation scenarios demonstrate that our VGFA framework achieves state-of-the-art performance, outperforming existing feature alignment methods in domain adaptive nuclei detection.