Are We Underestimating Overfitting?

Area of Science:

• Computational chemistry
• Machine learning
• Quantitative structure-activity relationships (QSAR)

Background:

• Traditional Quantitative Structure-Activity Relationship (QSAR) modeling emphasizes parsimonious models to avoid overfitting.
• Overfitting, where models perform well on training data but poorly on new data, is a key concern in model development.

Purpose of the Study:

• To challenge the dogma that parsimonious models are always superior in Quantitative Structure-Activity Relationship (QSAR) and Quantitative Structure-Property Relationship (QSPR) modeling.
• To explore the potential of overparameterized machine learning models to accurately predict external data.

Main Methods:

• Review of recent publications on overfitting and overparameterization in machine learning.
• Analysis of information theoretic arguments supporting the predictive power of overparameterized models.
• Modeling of synthetic and real data to evaluate the performance of potentially overfitted models on test sets.

Main Results:

• Formally overparameterized machine learning models can exhibit strong predictive accuracy on external test data.
• Supernumerary model parameters may contain valuable information that improves predictions for unseen data.
• Illustrative examples demonstrate the effectiveness of overfitted models in QSAR and QSPR.

Conclusions:

• The understanding of overfitting and overparameterization in machine learning has significant implications for QSAR and QSPR modeling.
• Overparameterized models offer a counterintuitive but effective approach to enhance predictive accuracy in cheminformatics.
• Embracing potentially overfitted models can lead to more robust and accurate structure-activity and structure-property relationship predictions.