Machine learning in computational chemistry: interplay between (non)linearity, basis sets, and dimensionality

Area of Science:

• Physical Chemistry
• Theoretical and Computational Chemistry
• Materials Chemistry

Background:

• Machine learning (ML) methods, including neural networks and kernel methods, are prevalent in physical, theoretical, computational, and materials chemistry.
• These techniques are applied to diverse problems like predicting material properties and constructing interatomic potentials.
• Often, advanced ML concepts are used without clear justification over simpler alternatives.

Purpose of the Study:

• To explore the interrelations between popular ML techniques and traditional linear regression and basis expansions.
• To identify limitations of current ML approaches, particularly in high-dimensional regimes.
• To propose strategies for enhancing ML methods' expressive power and interpretability.

Main Methods:

• Analysis of popular ML techniques (neural networks, kernel methods) in relation to linear models.
• Investigation of ML performance in high-dimensional scenarios.
• Discussion of high-dimensional model representation for hyperparameter selection and insight generation.

Main Results:

• Demonstration that certain ML approximations may fail in very high-dimensional settings.
• Identification of connections between nonlinear ML and traditional linear methods.
• Highlighting the potential collapse of ML approximations under specific conditions.

Conclusions:

• ML methods in chemistry, while powerful, require careful consideration of their underlying mathematical principles.
• Understanding the relationship between ML and linear models can prevent potential failures in high dimensions.
• Strategies exist to improve ML interpretability and robustness in chemical applications.