Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Experiment Videos

Subspace information criterion for model selection.

M Sugiyama1, H Ogawa

  • 1Department of Computer Science, Graduate School of Information Science and Engineering, Tokyo Institute of Technology, Meguro-ku, Tokyo, 152-8552, Japan.

Neural Computation
|August 17, 2001
PubMed
Summary

We introduce the subspace information criterion (SIC), a novel model selection method for supervised learning. SIC provides an unbiased estimate of generalization error, performing well even with limited training data.

Related Concept Videos

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Atopic dermatitis: studies of skin permeability and effectiveness of topical PUVA treatment.

Pediatric dermatology·1992
Same author

Stimulation of keratinocyte migration by growth factors.

The Journal of dermatology·1992
Same author

A wound healing model using healing-impaired diabetic mice.

The Journal of dermatology·1992
Same author

The response to drug therapy in unstable angina on the basis of coronary angiography findings.

Japanese circulation journal·1992
Same author

[The pathogenic role of Chlamydia trachomatis in otitis media with effusion].

Nihon Jibiinkoka Gakkai kaiho·1992
Same author

[Prospective randomized trials comparing hyperfractionated radiotherapy with conventional radiotherapy in stage III non-small cell lung cancer].

Nihon Igaku Hoshasen Gakkai zasshi. Nippon acta radiologica·1992

Area of Science:

  • Machine Learning
  • Statistical Learning Theory

Background:

  • Model selection is crucial for enhancing generalization in supervised learning.
  • Existing criteria like Mallows's C(L) have limitations.
  • Generalization error is often defined using Hilbert space norms.

Purpose of the Study:

  • To propose a new model selection criterion, the subspace information criterion (SIC).
  • To generalize Mallows's C(L) for improved model selection.
  • To provide an unbiased estimate of generalization error in a Hilbert space setting.

Main Methods:

  • Defining generalization error as the Hilbert space squared norm.
  • Developing SIC as a generalization of Mallows's C(L).
  • Proposing a practical calculation method for least-mean-squares learning.

Related Experiment Videos

Main Results:

  • SIC provides an unbiased estimate of the defined generalization error.
  • A calculation method is presented assuming Hilbert space dimension is less than the number of training examples.
  • Computer simulations demonstrate SIC's effectiveness with small datasets.

Conclusions:

  • The subspace information criterion (SIC) is a viable and effective method for model selection.
  • SIC offers improved generalization capability in supervised learning.
  • The proposed method shows promise, particularly in scenarios with limited training data.