HiAdapter: Histopathology-Induced Adapter for Pathology Foundation Models

With the rapid development of pathology foundation models, there is a growing demand for efficient fine-tuning strategies tailored to downstream tasks. However, existing parameter-efficient fine-tuning approaches are largely task-agnostic and exhibit limited generalization to histopathological images, particularly for unseen cancers and stains, due to substantial stain variability and the complexity of tissue microenvironments. To address these challenges, we present Histopathology-induced Adapter (HiAdapter), which incorporates domain-specific insights into staining and imaging mechanisms of histopathology. HiAdapter reconstructs stain-invariant representations via a Stain-invariant Adapter (S-Adapter) and integrates morphological features through a Morphology-aware Adapter (M-Adapter), effectively bridging the gap between low-level optical properties and high-level tissue semantics. Additionally, we introduce a Pathology Prototypical Contrastive Loss (PPCLoss) to reduce inter-class similarity and mitigate intra-class heterogeneity, enhancing feature discriminability. Extensive experiments using three pathology foundation models (CTransPath, CONCH and UNI) across six benchmarks, including two public datasets, an osteosarcoma tissue classification dataset (56,178 patches) and a chondrosarcoma necrosis classification dataset (3,867 patches) for unseen cancers generalization, as well as an IHC-stained dataset (4,967 patches) and an HIF1A IHC-stained dataset (4,433 patches) for unseen stains generalization, demonstrate the effectiveness of HiAdapter in both efficiency and accuracy. HiAdapter achieves an average improvement of 2.15 in F1 and 1.55 in accuracy over the second-best performer, maintaining strong biological and diagnostic interpretability. External validation on an independent osteosarcoma dataset (9,535 patches) and WSI-level survival analysis (178 slides) further confirm the superior generalizability and underscore the potential for patient-level diagnosis and prognosis in clinical practice. Our code is available at https://github.com/idata-ora/HiAdapter.