Trust Beyond Accuracy: Conformal Uncertainty Quantification Reveals the Generalization Gap in Polymer Glass...

Area of Science:

• Polymer Informatics
• Machine Learning
• Computational Materials Science

Background:

• Glass transition temperature (Tg) is a critical thermophysical property for polymers.
• Existing machine learning (ML) studies often prioritize point prediction accuracy over reliability under chemical novelty.

Purpose of the Study:

• To evaluate the reliability of established ML models (XGBoost, SVR) for polymer Tg prediction using simulation-derived data.
• To assess the impact of chemical novelty on model performance and reliability using various validation strategies.
• To investigate the utility of conformal prediction intervals for uncertainty quantification and triage.

Main Methods:

• Utilized a 410-sample simulation-derived polymer dataset.
• Employed stratified, scaffold-based, and fingerprint-clustered validation regimes.
• Combined learning curves, applicability domain diagnostics, split conformal prediction (SCP), subgroup analysis, and descriptor importance analysis.

Main Results:

• Model performance degraded and variability increased under novelty-enforcing splits.
• Support vector regression (SVR) exhibited more stable point-prediction behavior than gradient-boosted trees (XGBoost) in certain regimes.
• Conformal intervals maintained coverage but were often too wide for precise ranking, revealing local reliability limitations.

Conclusions:

• A reproducible workflow for assessing ML reliability in polymer Tg prediction was established.
• Conformal intervals serve as risk indicators for uncertainty-aware triage, not high-resolution screening tools.
• Experimental validation is crucial before deploying models for measured Tg data.