Hierarchical Tissue-Specific Modeling of Pathology Images Predicts Response in HER2+ Breast Cancer

The structural and spatial heterogeneity of the tumor microenvironment in human epidermal growth factor receptor 2-positive (HER2+) breast cancer (HER2+BC) poses major challenges for predicting pathologic complete response (pCR) to neoadjuvant chemotherapy. While immune-related biomarkers from whole-slide images have been widely explored, tissue-level interaction patterns and semantic features remain underexplored, particularly in characterizing structural organization within tissue compartments. To address this, we propose an interpretable hierarchical framework that models the tissue-specific organization of graph-based structural features and deep-learning-derived pCR scores (DLPSs) and integrates clinical variables to enhance pCR prediction. Whole-slide images were segmented into 5 biologically distinct compartments: tumor, stroma, stromal tumor-infiltrating lymphocytes (sTILs), intratumoral tumor-infiltrating lymphocytes (iTILs), and combined tumor-infiltrating lymphocytes (sTILs plus iTILs). Each compartment was modeled as a graph in which tile-level cluster centers served as nodes and their connections as edges. Structural features were computed using social network analysis (SNA) metrics to characterize spatial organization in each tissue compartment. In parallel, DLPSs were generated using a pretrained clustering-constrained multiple-instance learning model as a feature extractor, followed by training tissue-specific multilayer perceptron classifiers. The tissue-specific SNA features, DLPSs, and clinical variables were integrated for prediction. The model was trained on the Yale Response dataset and externally validated using the IMage-based Pathological REgistration and Segmentation Statistics (IMPRESS) HER2+ dataset. Across compartments, stroma achieved the highest predictive performance (area under the receiver operating characteristic curve [AUC] = 0.907), surpassing a reported method by 9.5%. Notably, SNA features achieved an AUC of 0.793, outperforming DLPS (0.596) and clinical variables (0.757). These findings suggest the value of integrating tissue-specific structural and semantic features for interpretable modeling of treatment response variability in HER2+BC.