A Novel Self-Attention Mechanism-Based Dynamic Ensemble Model for Soil Hyperspectral Prediction

Area of Science:

• Soil Science
• Spectroscopy
• Machine Learning

Background:

• Visible-near-infrared (VNIR) spectroscopy offers rapid, non-destructive soil organic matter (SOM) detection.
• Accurate SOM prediction hinges on effective algorithmic models, particularly Weighted Averaging Ensemble Models (WAEM).
• Standard WAEM methods struggle with optimal base learner weight allocation, leading to biased weights and suboptimal performance.

Purpose of the Study:

• To enhance WAEM performance for SOM prediction by optimizing base learner weights.
• To investigate novel methods for dynamic weight allocation using training process information.
• To evaluate the efficacy of a self-attention mechanism (Sam) in improving WAEM accuracy.

Main Methods:

• Developed seven methods for optimal WAEM weight determination using training process information, including reinforcement learning and Sam.
• Utilized a dynamic data structure for real-time weight updating with 704 soil samples from the Tarim River Basin.
• Compared performance against conventional evaluation index approaches.

Main Results:

• Six WAEM methods incorporating training process information outperformed conventional methods.
• WAEM root mean square error (RMSE) decreased by 0.028-1.279 g kg^-1, and the correlation coefficient (R²) increased by up to 0.06.
• Sam achieved the highest performance (R²=0.927, RMSE=2.325 g kg^-1), with model R² converging at 26 base learners.

Conclusions:

• Dynamic WAEM weight allocation using training process information, particularly Sam, significantly improves SOM prediction accuracy.
• This approach provides a robust foundation for advancing infrared-based soil monitoring.
• Optimizing base learner weights is crucial for maximizing the potential of ensemble models in spectroscopic analysis.