Beyond Random Splits: Assessing the Generalization of Graph and Vector Models for WT-Structure-Only Drug Resistance...

Background: Predicting mutation-induced changes in binding free energy (ΔΔG) is important for understanding drug resistance and prioritizing resistant variants, yet real-world generalization remains unclear. In clinical diagnosis and early-stage drug screening, mutant complex structures are often unavailable. However, many existing methods rely on paired wild-type (WT) and mutant structures and are evaluated under random splits that may permit substantial protein-level train-test overlap. Methods: We established a WT-complex-structure-only setting, in which mutant structural coordinates are unavailable but mutation annotations are provided, and conducted a controlled comparison of graph- and vector-based configurations under both random and strict UniProt-based splits. We further analyzed graph context, message passing, representation bias, and dataset noise to identify factors limiting protein-disjoint generalization. Results: On MdrDB, random splits yielded apparently moderate performance (Pearson R ≈ 0.55), whereas strict UniProt-based evaluation simulating unseen proteins led to a marked drop (Pearson R ≈ 0.15), indicating that random splits substantially overestimate generalization. Graph-based modeling retained a weak but nonzero signal relative to the vector baseline, although the differences were limited and not statistically significant. Ablation analyses suggested that full-protein structural context was more useful than local pocket-plus-mutation context, whereas full-protein message passing did not provide a clear additional advantage. ΔESM representations reduced protein-background clustering, but overall strict-split performance remained low. Conclusions: WT-only prediction of mutation-induced drug resistance from static structures remains far from solved under realistic protein-disjoint evaluation. Experimental label inconsistency, sparse protein coverage, and missing dynamic structural information may further limit performance, underscoring the need for split-aware evaluation protocols and stronger physical priors.