Search research articles

ABOUT JoVE

Overview Leadership Blog JoVE Help Center

AUTHORS

Publishing Process Editorial Board Scope & Policies Peer Review FAQ Submit

LIBRARIANS

Testimonials Subscriptions Access Resources Library Advisory Board FAQ

RESEARCH

JoVE Journal Methods Collections JoVE Encyclopedia of Experiments Archive

EDUCATION

JoVE Core JoVE Business JoVE Science Education JoVE Lab Manual Faculty Resource Center Faculty Site

Terms & Conditions of Use

Search research articles

Related Experiment Videos

On cross validation for model selection.

I Rivals¹, L Personnaz

¹Laboratoire d'Electronique, Ecole Superieure de Physique et de Chimie Industrielles (ESPCI), 10 rue Vauquelin, 75231, Paris, France. Isabelle.Rivals@espci.fr

Neural Computation

|May 5, 1999

Summary

Cross-validation performs poorly for linear model selection compared to statistical tests. Statistical tests are recommended over cross-validation for both linear and nonlinear model selection.

Related Concept Videos

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by

Same author

Specific postural alignment alterations due to long-face deformity in patients with maxillo-mandibular deformities.

Journal of stomatology, oral and maxillofacial surgery·2024

Same author

Neuroergonomic and psychometric evaluation of full-face crew oxygen masks respiratory tolerance: a proof-of-concept study.

Journal of the Royal Army Medical Corps·2018

Same author

Modifications of the endosomal compartment in peripheral blood mononuclear cells and fibroblasts from Alzheimer's disease patients.

Translational psychiatry·2015

Same author

Klf9 is necessary and sufficient for Purkinje cell survival in organotypic culture.

Molecular and cellular neurosciences·2012

Same author

Nonlinear internal model control using neural networks: application to processes with delay and design issues.

IEEE transactions on neural networks·2008

Same author

No free lunch with the sandwich [sandwich estimator].

IEEE transactions on neural networks·2008

Area of Science:

Statistics
Machine Learning
Data Science

Background:

Leave-one-out cross-validation was recently shown to not be subject to the "no-free-lunch" criticism.
Model selection is a critical step in statistical analysis and machine learning.

Purpose of the Study:

To evaluate the performance of cross-validation for linear model selection.
To compare cross-validation with classic statistical tests for model selection.

Main Methods:

The study compared the performance of cross-validation with classic statistical tests.
The comparison focused on the selection of linear models.

Main Results:

Cross-validation demonstrated very poor performance in selecting linear models.
Classic statistical tests outperformed cross-validation for linear model selection.

Conclusions:

Statistical tests are preferable to cross-validation for selecting linear models.
Statistical tests are also recommended for nonlinear model selection.

Related Experiment Videos