PDTSyn: a parameter-decomposed transformer for domain-generalized cell line-aware drug synergy prediction

Drug combination therapy is a key strategy in cancer treatment, and accurately predicting synergistic drug pairs is crucial for improving therapeutic efficacy. While machine learning methods have advanced this task, their performance is often limited by two challenges: the strong cell line specificity of drug synergy and the poor generalization of models to unseen cellular contexts. Existing approaches tend to emphasize cell line-specific modeling but struggle to generalize across diverse biological domains. We propose PDTSyn, a domain generalization-driven framework that addresses these limitations through disentangled representation learning. PDTSyn treats each cell line as a distinct domain and separates drug representations into domain-invariant and domain-specific components. The parameter-decomposed transformer dynamically generates cell line-adaptive attention parameters from cell features, enabling flexible modeling of cell-specific drug-drug interactions while preserving shared pharmacological structure. To further enhance generalization, we introduce a dual regularization strategy: a cross-domain Kullback-Leibler-divergence loss that aligns invariant embeddings across cell lines, and a cell-line discriminative loss that enforces the specificity of domain-dependent representations. Comprehensive experiments on the O'Neil and NCI-ALMANAC datasets demonstrate that PDTSyn consistently outperforms state-of-the-art baselines under standard evaluation protocols. Moreover, PDTSyn maintains strong performance in challenging unseen cell line, unseen drug, and unseen drug pair settings, highlighting its robustness to distribution shifts. These results indicate that explicitly disentangling invariant and specific mechanisms provides an effective and generalizable solution for drug synergy prediction in heterogeneous biological environments.