Language-guided multimodal domain generalization for outcome prediction of head and neck cancer

Accurate prediction of head and neck cancer recurrence across medical institutions remains challenging due to inherent domain shifts in imaging data. Current domain generalization methods primarily focus on learning domain-invariant features from medical images, often overlooking structured clinical information that inherently exhibits cross-institutional consistency. To leverage clinical data and enhance the model's generalization, we propose an end-to-end Language-Guided Multimodal Domain Generalization (LGMDG) method. LGMDG is composed of two main components: the first is a Language-guided clinical domain-invariant feature extraction module designed with a language-based clinical prompt to effectively extract continuous semantic information. The second is a contrastively and adversarially enhanced multimodality classification network of three branches. The multimodal factorized bilinear pooling classifier is responsible for predicting the outcome. The other two branches utilize domain adversarial learning and cross-modal alignment to facilitate domain-invariant representations during training, thus improving the model's generalization capability. Extensive experiments are performed on the six cancer centers dataset through leave-one-domain-out validation. Our proposed LGMDG achieves an average accuracy of 81.04% and an average AUC of 76.91% across the six domains, with 24.83M parameters and 17.36 GFLOPs. These results demonstrate that our method achieves superior generalization performance while maintaining nearly the same level of complexity compared to eleven competing approaches. The code and trained models were released on GitHub.¹.