Demystifying uncertainty in PM10 susceptibility mapping using variable drop-off in extreme-gradient boosting (XGB)...

Area of Science:

• Environmental Science
• Data Science
• Geospatial Analysis

Background:

• Machine learning models can exhibit uncertainty due to variable importance variance and imbalanced data effects on prediction accuracy.
• The Receiver Operating Characteristic (ROC) curve may not always reflect the true performance with unbalanced prediction variables.

Purpose of the Study:

• To investigate and mitigate uncertainty in machine learning predictions, particularly concerning variable importance and model generalization.
• To develop and test a variable drop-off loop function for improving model performance and reliability.

Main Methods:

• Modeled a susceptibility index for airborne particulate matter (PM10) using Landsat 8 imagery spectral bands/indices and OpenStreetMap data.
• Employed a variable drop-off loop function incorporating early termination, regularization, and generalization control.
• Utilized extreme-gradient boosting (XGBOOST) and random forest (RF) algorithms to generate probability and classification index maps.

Main Results:

• Assessed model performance using criteria including overall accuracy, variable quantity, processing time, overfitting, importance distribution, and Area Under the ROC Curve (AUC).
• The variable drop-off loop function was evaluated for its effectiveness in reducing model capacity and controlling generalization.
• Compared the performance of XGBOOST and RF algorithms in the context of PM10 susceptibility mapping.

Conclusions:

• The tested variable drop-off function shows potential for managing machine learning uncertainty and improving the reliability of environmental prediction models.
• The study highlights the importance of evaluating multiple utility criteria beyond ROC for robust model assessment.
• Findings contribute to more accurate geospatial modeling of air quality indicators like PM10.