Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Experiment Videos

Benign overfitting in linear regression.

Peter L Bartlett1,2, Philip M Long3, Gábor Lugosi4,5,6

  • 1Department of Statistics, University of California, Berkeley, CA 94720-3860; peter@berkeley.edu.

Proceedings of the National Academy of Sciences of the United States of America
|April 26, 2020
PubMed
Summary

Related Concept Videos

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Gradient Descent with Identity Initialization Efficiently Learns Positive-Definite Linear Transformations by Deep Residual Networks.

Neural computation·2019
Same journal

The TaMYB55-TaSnRK1α1-TabZIP9 module confers heat stress tolerance in wheat.

Proceedings of the National Academy of Sciences of the United States of America·2026
Same journal

Superstatistics approach to turbulent circulation fluctuations.

Proceedings of the National Academy of Sciences of the United States of America·2026
Same journal

A molecular timescale for evolution of cobamide biosynthesis.

Proceedings of the National Academy of Sciences of the United States of America·2026
Same journal

Pierre Chambon, a pioneer of molecular biology and gene regulation in eukaryotes.

Proceedings of the National Academy of Sciences of the United States of America·2026
Same journal

Granulosa cell glycogen fuels the avascular corpus luteum.

Proceedings of the National Academy of Sciences of the United States of America·2026
Same journal

Synthetic essentiality of TRAIL/TNFSF10 in VHL-deficient renal cell carcinoma.

Proceedings of the National Academy of Sciences of the United States of America·2026
See all related articles
This summary is machine-generated.

Benign overfitting, where models fit noisy data well, is explained in linear regression. Overparameterization is key, requiring many unimportant parameters to exceed the sample size for accurate predictions.

Area of Science:

  • Machine Learning
  • Statistical Learning Theory

Background:

  • Deep learning models exhibit benign overfitting, accurately predicting outcomes despite fitting noisy training data.
  • This phenomenon, where models generalize well with perfect data fit, remains a key mystery in machine learning.

Purpose of the Study:

  • To characterize linear regression problems where minimum norm interpolation achieves near-optimal prediction accuracy.
  • To investigate the role of overparameterization and data dimensionality in benign overfitting.

Main Methods:

  • Analysis of linear regression with a focus on minimum norm interpolating predictors.
  • Characterization based on effective rank of the data covariance matrix.
  • Comparative study of data in finite vs. infinite-dimensional spaces.
Keywords:
interpolationlinear regressionoverfittingstatistical learning theory

Related Experiment Videos

Main Results:

  • Overparameterization is essential for benign overfitting in linear regression.
  • Accurate prediction requires the number of unimportant parameter directions to significantly exceed the sample size.
  • Finite-dimensional data with growing dimensions enhances the accuracy of minimum norm interpolation compared to infinite-dimensional data.

Conclusions:

  • Benign overfitting in linear regression is understood through a characterization involving effective data rank and overparameterization.
  • The dimensionality of the data space critically influences the conditions under which minimum norm interpolation achieves high accuracy.