Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Experiment Videos

Universal learning curves of support vector machines.

M Opper1, R Urbanczik

  • 1Department of Computer Science and Applied Mathematics, Aston University, Birmingham B4 7ET, United Kingdom.

Physical Review Letters
|May 1, 2001
PubMed
Summary

Support vector machines (SVMs) with infinitely complex kernels can achieve optimal generalization error on noisy data, challenging prior beliefs. Learning curves in SVMs exhibit universal behavior, independent of kernel choice but dependent on the target rule.

Related Concept Videos

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Inferring hidden states in Langevin dynamics on large networks: Average case performance.

Physical review. E·2017
Same author

Laboratory tests focusing on sputum.

The international journal of tuberculosis and lung disease : the official journal of the International Union against Tuberculosis and Lung Disease·2010
Same author

Scaling up tuberculosis culture services: a precautionary note.

The international journal of tuberculosis and lung disease : the official journal of the International Union against Tuberculosis and Lung Disease·2009
Same author

Region growing with pulse-coupled neural networks: an alternative to seeded region growing.

IEEE transactions on neural networks·2008
Same author

Enumerating constrained elementary flux vectors of metabolic networks.

IET systems biology·2007
Same author

Yield of serial sputum specimen examinations in the diagnosis of pulmonary tuberculosis: a systematic review.

The international journal of tuberculosis and lung disease : the official journal of the International Union against Tuberculosis and Lung Disease·2007

Area of Science:

  • Statistical physics
  • Machine learning theory
  • Computational learning theory

Background:

  • Support Vector Machines (SVMs) are a powerful class of supervised learning algorithms.
  • Model complexity is a critical factor influencing the generalization performance of machine learning models.
  • Understanding the learning dynamics of SVMs, especially with complex kernels, is crucial for advancing artificial intelligence.

Purpose of the Study:

  • To investigate the impact of model complexity on the learning capabilities of Support Vector Machines (SVMs).
  • To analyze the generalization error of SVMs with kernels of infinite complexity when applied to noisy target rules.
  • To identify universal characteristics of learning curves in SVMs.

Main Methods:

  • Application of statistical physics methodologies to analyze learning in SVMs.

Related Experiment Videos

  • Theoretical investigation of SVMs with kernels of infinite complexity.
  • Analysis of generalization and training errors in the context of noisy target rules.

    • Main Results:

    • SVMs with infinite kernel complexity achieve optimal generalization error on noisy target rules, contrary to common theoretical assumptions.
    • Training error does not necessarily converge to generalization error even with optimal generalization.
    • A universal asymptotic behavior of learning curves was discovered, dependent solely on the target rule and not the SVM kernel.

    Conclusions:

    • Infinite model complexity in SVMs can be advantageous for learning from noisy data.
    • The findings challenge conventional wisdom regarding the relationship between training error, generalization error, and model complexity.
    • The universality of learning curves offers a simplified framework for understanding SVM learning dynamics.