A Robust Gaussian Process Paradigm for Predictive Modeling on Small Data sets in Environmental Science: A Case Study...

Area of Science:

• Environmental Science
• Machine Learning
• Data Science

Background:

• Environmental process optimization faces challenges from complex interactions and limited data, leading to overfitted machine learning (ML) models.
• Traditional ML models often struggle with generalization on small, heterogeneous experimental datasets common in environmental research.

Purpose of the Study:

• To develop a robust machine learning framework, Gaussian Process Bayesian Tuning (GP-BT), for optimizing environmental processes.
• To enhance the generalization and real-world performance of ML models using small, complex environmental datasets.

Main Methods:

• Developed GP-BT, a Gaussian Process Bayesian Tuning framework optimizing kernel selection and hyperparameters by minimizing cross-validation loss.
• Evaluated GP-BT against conventional algorithms (Random Forest, XGBoost, CatBoost) and standard Gaussian Process models on three environmental datasets.
• Validated GP-BT through 52 laboratory experiments and analyzed model interpretability using SHapley Additive exPlanations (SHAP).

Main Results:

• GP-BT demonstrated superior robustness and generalization compared to conventional ML algorithms and standard Gaussian Process models.
• The framework achieved lower prediction errors on unseen conditions in laboratory experiments.
• GP-BT identified optimal conditions for combined sewer overflows treatment, reaching 98% removal efficiency, significantly outperforming an overfitted Random Forest model (89%).

Conclusions:

• GP-BT provides a reliable framework for extracting insights from costly, small-scale environmental experimental data.
• The method's conservative learning strategy is crucial for robust performance with sparse, noisy data.
• GP-BT accelerates the discovery of hidden performance potential in environmental technologies, supported by an open-source package and web platform.