Ensemble transformer-based multiple instance learning for predicting neoadjuvant chemotherapy response from breast
Zhenshui Wu1, Kaining Ye2, Jianming Weng2
1Department of Pathology, The Second Affiliated Hospital of Fujian Medical University, Quanzhou, Fujian, China.
Introduction:
Neoadjuvant chemotherapy (NAC) is a cornerstone of breast cancer management, and accurate prediction of therapeutic efficacy is essential for optimizing treatment strategies and improving patient outcomes.
Methods:
This study proposes an integrated Transformer-based Multiple Instance Learning (MIL) framework that leverages pre-treatment biopsy whole-slide images (WSIs) to predict NAC response. A multi-institutional dataset of 128 patients was collected, comprising 86 cases for training, 42 for internal validation, and 22 microscope images for external validation. The framework integrates ResNet50 feature extraction, a multi-scale attention Transformer encoder, and a two-stage classification strategy to capture both local morphological and global contextual features. Class imbalance was mitigated using SMOTE and ADASYN, while domain adaptation (DANN) and metric learning enhanced cross-modal robustness.
Results:
The proposed model achieved a WSI-level accuracy of 79.3% in internal validation and demonstrated strong discriminative ability in identifying pathological complete response (pCR, AUC = 0.82) and non-response (AUC = 0.77). External validation using lower-resolution microscope images yielded an AUC of 0.70 for pCR and 0.67 for non-response, outperforming traditional CNN architectures such as GoogleNet, ResNet34, and SqueezeNet. The model's heatmap visualizations revealed well-defined lesion boundaries and interpretable regions of interest, underscoring its clinical transparency.
Discussion:
By relying solely on hematoxylin and eosin-stained WSIs, the framework provides a fully automated, interpretable, and resource-efficient approach suitable for real-world deployment. The two-stage classification design offers fine-grained stratification between pCR, partial, and poor responders, which is critical for personalized therapy planning. Future work will focus on expanding multi-center datasets and integrating advanced pathology foundation models to further enhance cross-domain generalization and clinical applicability.
