Knowledge-guided Multimodal Learning for Interpretable Prediction of Pathological Complete Response and Survival in...

Pretreatment estimation of pathological complete response (pCR) to neoadjuvant chemotherapy may support risk stratification and future response-adapted clinical studies in breast cancer. However, existing deep learning approaches often rely on "black-box" visual pattern recognition, suffering from poor interpretability, high data dependency, and limited generalizability across heterogeneous cohorts. Here, we present CLASP (Clinical Language-Aligned Slide Pathology), a knowledge-driven, multimodal framework that synergizes pathological expert knowledge with vision-language foundation models.

CLASP integrates whole slide image (WSI) features encoded by CONCH, patient-specific clinical data, and 16 hierarchical pathological text priors generated by an ensemble of Large Language Models. Through a novel vision-language feature aggregation module and a multi-task consistency constraint, the model aligns visual biomarkers with semantic concepts to jointly predict pCR and prognosis.

Validated on 1,486 patients from four independent cohorts across multiple medical centers, CLASP achieved strong discriminative performance, yielding an Area Under the Curve (AUC) of 0.881 in internal testing and maintaining robustness in external validation (AUC 0.875 and 0.845), with higher AUC point estimates than the 11 evaluated baseline methods. Notably, CLASP also showed improved data efficiency, achieving competitive performance with as few as 8 training samples (few-shot learning). Interpretability analyses suggested that model-highlighted regions were consistent with recognized pathological features associated with treatment response, such as tumor-infiltrating lymphocytes and stromal architecture.

These findings suggest that pathology-informed multimodal learning may provide a candidate framework for pretreatment risk stratification.