QUAD: a composite risk framework integrating uncertainty, applicability domain, and model disagreement for reliable...

Reliable prediction confidence estimation remains a major challenge in quantitative structure-activity relationship (QSAR) modeling, particularly when models are applied to structurally novel compounds or heterogeneous experimental datasets. Although several uncertainty quantification approaches have been proposed, the relative behavior of complementary reliability indicators under extrapolative validation conditions remains insufficiently characterized. In the present study, we introduce QUAD, a post-hoc reliability estimation framework integrating ensemble-based uncertainty, model disagreement, and applicability-domain (AD) risk into a unified molecule-level reliability score. The framework was evaluated using Random Forest regression and ECFP4 molecular fingerprints on two Alzheimer's disease-related benchmark datasets targeting β-site amyloid precursor protein cleaving enzyme 1 (BACE1) and glycogen synthase kinase-3β (GSK3β). Reliability behavior was systematically assessed under both conventional random train-test validation and more stringent Bemis-Murcko scaffold-based validation. In addition to evaluating the integrated QUAD framework, component-wise ablation analysis and reliability enrichment analysis were performed to investigate the relative contributions of uncertainty, disagreement, and applicability-domain information. Under conventional random-split evaluation, ensemble uncertainty demonstrated the strongest and most consistent correlation with prediction error across both datasets. For BACE1, uncertainty achieved Pearson and Spearman correlations of 0.366 and 0.338, respectively, while the integrated QUAD framework produced lower correlations (Pearson r = 0.303, Spearman ρ = 0.244). In contrast, GSK3β exhibited stronger overall reliability behavior, with QUAD retaining substantial correlation with prediction error (Pearson r = 0.439). Scaffold-based evaluation revealed pronounced dataset-dependent differences. Under scaffold extrapolation, QUAD performance deteriorated substantially in BACE1 (Pearson r = 0.071), whereas GSK3β retained comparatively robust reliability behavior (Pearson r = 0.363). Ablation analyses further demonstrated that ensemble uncertainty represented the most stable reliability indicator across validation regimes, while disagreement-based reliability estimation exhibited substantial sensitivity to structural extrapolation. Taken together, the present results indicate that integrated reliability estimation frameworks should not be interpreted as universally superior to individual uncertainty estimation approaches. Instead, the effectiveness of composite reliability estimation appears strongly dependent on dataset-specific relationships between constituent reliability indicators and validation conditions. The study further highlights the importance of scaffold-based evaluation for realistic assessment of molecule-level reliability estimation strategies in practical QSAR deployment scenarios.