Functional form estimation using oblique projection matrices for LS-SVM regression models.

Area of Science:

• Machine Learning
• Statistical Modeling
• Data Science

Background:

• Kernel regression models are powerful non-parametric tools for data fitting.
• Their "black box" nature obscures the specific relationships between input variables and the output.
• Understanding these relationships is crucial for model interpretability and physical meaning.

Purpose of the Study:

• To develop a novel methodology for retrieving and interpreting the functional relations within least squares support vector machine (LS-SVM) regression models.
• To decompose the LS-SVM model's output into partial nonlinear contributions and interaction effects of input variables.
• To provide a tool for understanding and physically interpreting regression models without prior knowledge.

Main Methods:

• The proposed method, Nonlinear Oblique Subspace Projectors (NObSP), utilizes oblique subspace projectors (ObSP) to decouple regressor influences.
• It estimates functional relations through nonlinear transformations and kernel evaluations.
• The output is decomposed into additive partial contributions and interaction terms.

Main Results:

• NObSP successfully retrieves and estimates the functional relations between input regressors and the output.
• The methodology demonstrated superior performance compared to the component selection and smooth operator (COSSO) method.
• Stable estimations of functional relations were achieved on both synthetic and real-world datasets (concrete strength).

Conclusions:

• NObSP offers a robust approach to interpret complex kernel regression models.
• It provides stable and accurate estimations of input-output functional relationships.
• This method enhances the physical interpretability of regression models, valuable for scientific and engineering applications.