Moving beyond generalization to accurate interpretation of flexible models

Area of Science:

• Computational Neuroscience
• Machine Learning in Science

Background:

• Flexible machine learning models are increasingly used in science for data prediction.
• Interpreting these models to derive scientific hypotheses is a growing area of interest.
• The relationship between predictive accuracy and interpretability in these models remains unclear.

Purpose of the Study:

• To investigate whether high data prediction accuracy in flexible models guarantees accurate scientific interpretation.
• To develop a method for identifying models with correct interpretations, separating true features from noise.

Main Methods:

• Utilized a flexible and intrinsically interpretable framework for modeling neural dynamics.
• Compared model features across data samples to differentiate signal from noise.
• Tested the approach using neural recordings from the visual cortex of behaving monkeys.

Main Results:

• Many optimized models achieved high data prediction but failed to reflect the correct underlying hypothesis.
• The developed alternative approach successfully identified models with accurate interpretations.
• Demonstrated that predictive performance does not equate to interpretability.

Conclusions:

• Accurate interpretation of flexible models is not guaranteed by good data prediction.
• Proposed a principled approach to select models that provide correct scientific interpretations.
• Highlighted the importance of interpretability for hypothesis generation in machine learning applications.