Rethinking local spectral modelling: From per-query refitting to model libraries.

Area of Science:

• Geospatial analysis
• Chemometrics
• Machine learning for spectroscopy

Background:

• Diffuse reflectance spectroscopy (DRS) libraries are complex and computationally intensive.
• Conventional methods require per-query model refitting, limiting scalability and interpretability.

Purpose of the Study:

• Introduce liblex, an algorithmic framework to streamline spectral library utilization.
• Convert DRS libraries into pre-computed localized models (experts) for efficient retrieval and combination.

Main Methods:

• Developed liblex to transform DRS libraries into expert models.
• Evaluated liblex on a cross-continental soil total carbon (TC) prediction task.
• Compared liblex performance against benchmark methods including LOCAL, Cubist, PLS, and CNN with transfer learning.

Main Results:

• liblex achieved high predictive accuracy in a challenging cross-continental test case.
• Demonstrated lower RMSE values compared to benchmark methods for soil TC prediction.
• Showcased liblex's ability to provide uncertainty proxies (prediction dispersion) and interpretability (regression coefficients, variable importance).

Conclusions:

• liblex overcomes limitations of conventional local modeling by using pre-computed experts, reducing computational load.
• Enables deployment without the full reference library, preserving interpretability and supporting robust predictions across diverse domains.
• Offers a scalable, interpretable, and privacy-preserving strategy for operationalizing large spectral libraries.