When spectral libraries are too complex to search: Evolutionary subset selection for domain-adaptive calibration

The increasing availability of large spectral libraries offers new opportunities to reduce the costs and efforts required to develop spectroscopy-based sensing techniques for rapidly and non-destructively estimating key properties across environmental and agricultural matrices. These libraries can provide training samples for developing models adapted to specific target domains. However, identifying which samples are most relevant for a given target domain remains challenging. This study introduces gesearch, a non-linear evolutionary algorithm for selecting target-domain-relevant training samples from complex spectral libraries to build accurate and interpretable quantitative models.

The gesearch, method was used to extract a subset of relevant samples from a large North American infrared soil spectral library to build predictive models of total carbon for an independent target area in the Democratic Republic of the Congo. In this challenging cross-domain test case, simple linear models built with the training samples found by gesearch achieved superior accuracy compared with established modelling approaches, including LOCAL, Cubist, convolutional neural networks, and global partial least squares regression.

The proposed method provides a practical framework for exploiting large heterogeneous spectral libraries when only limited target-domain reference data are available. By selecting samples that are spectrally and compositionally coherent with the target domain, gesearch supports accurate, compact, and interpretable calibration models. It can also operate when only unlabelled target-domain spectra are available.