Enhancing spectral interpretability and band selection prior to prediction model development via CAKE

Area of Science:

• Chemometrics
• Machine Learning
• Spectroscopy

Background:

• Machine learning models often lack causal interpretability, hindering the reliability of spectral data analysis.
• Spurious correlations in spectral data can lead to inaccurate prediction models.
• A direct causal link between spectral data and objective variables is crucial for robust models.

Purpose of the Study:

• To introduce the Causal Analysis via Kernel Estimation (CAKE) framework for enhancing causal interpretability in spectral data analysis.
• To develop a method that identifies single-component spectral bands and distinguishes true causal relationships from spurious correlations.
• To validate the CAKE framework on both simulated and real-world spectral datasets.

Main Methods:

• CAKE utilizes an information-theoretic approach, calculating mutual information differences between variables and regression residuals to determine causal direction.
• A Kernel Density Estimation (KDE) classifier is employed to differentiate causal structures prone to spurious correlations.
• The framework was optimized using simulated data and validated on near-infrared and fluorescence spectroscopy measurements of solvent mixtures.

Main Results:

• CAKE successfully characterized three types of causality: single direct cause, hidden causes, and confounders, using density functions.
• The framework accurately classified single-component spectral bands into their respective causal structures.
• CAKE identified reliable spectral bands, excluding spurious correlations, across diverse spectroscopic applications.

Conclusions:

• CAKE provides a pre-calibration method that enhances causal reliability and predictive performance in spectral data analysis.
• The framework operates independently of the calibration process and requires no prior knowledge of pure spectra.
• CAKE demonstrates that causal interpretability and practical performance in spectral analysis are achievable simultaneously.