Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Experiment Videos

Learning eigenfunctions links spectral embedding and kernel PCA.

Yoshua Bengio1, Olivier Delalleau, Nicolas Le Roux

  • 1Département d'Informatique et Recherche Opérationnelle, Centre de Recherches Mathématiques, Université de Montréal, Montréal, Québec, H3C 3J7, Canada. bengioy@iro.umontreal.ca

Neural Computation
|August 31, 2004
PubMed
Summary
This summary is machine-generated.

Related Concept Videos

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Imagining and building wise machines: the centrality of AI metacognition.

Trends in cognitive sciences·2026
Same author

Navigating Ternary Doping in Li-ion Cathodes With Closed-Loop Multi-Objective Bayesian Optimization.

Advanced materials (Deerfield Beach, Fla.)·2026
Same author

Divergent creativity in humans and large language models.

Scientific reports·2026
Same author

Identifying indicators of consciousness in AI systems.

Trends in cognitive sciences·2025
Same author

Publisher Correction: Deep-learning-based virtual screening of antibacterial compounds.

Nature biotechnology·2025
Same author

Deep-learning-based virtual screening of antibacterial compounds.

Nature biotechnology·2025
Same journal

A Model-Free Reinforcement Learning Implementation of Decision Making Under Uncertainty by Sequential Sampling.

Neural computation·2026
Same journal

DROP: Distributional and Regular Optimism and Pessimism for Reinforcement Learning.

Neural computation·2026
Same journal

Hierarchical Active Inference Using Successor Representations.

Neural computation·2026
Same journal

W-Kernel and Its Principal Space for Frequentist Evaluation of Bayesian Estimators.

Neural computation·2026
Same journal

A Hidden Markov Model-Inspired Sequence Classification Method for Hyperdimensional Computing.

Neural computation·2026
Same journal

Sparse Graphical Modeling for Electrophysiological Phase-Based Connectivity Using Circular Statistics.

Neural computation·2026
See all related articles

This study reveals spectral embedding methods and kernel principal components analysis are linked. A new approach extends these methods for out-of-sample data, improving generalization performance in machine learning.

Area of Science:

  • Machine Learning
  • Data Science
  • Dimensionality Reduction

Background:

  • Spectral embedding methods and kernel principal components analysis (KPCA) are widely used for dimensionality reduction.
  • Existing methods often struggle with generalizing to new, unseen data points.
  • Understanding the theoretical underpinnings of these methods is crucial for improving their performance.

Purpose of the Study:

  • To establish a direct relationship between spectral embedding methods and KPCA.
  • To present a unified framework for understanding these methods as special cases of a more general learning problem.
  • To develop a principled method for extending spectral embedding techniques to handle out-of-sample data.

Main Methods:

  • Formulating the learning problem as finding principal eigenfunctions of an operator defined by a kernel and the data-generating density.

Related Experiment Videos

  • Deriving the Nyström formula as a principled extension for out-of-sample examples.
  • Defining a loss function for spectral embedding methods and analyzing its expected value.
  • Conducting experiments using Locally Linear Embedding (LLE), Isomap, spectral clustering, and Multidimensional Scaling (MDS).

    • Main Results:

    • A direct relationship between spectral embedding and KPCA is demonstrated.
    • The Nyström formula is shown to be a valid extension for out-of-sample data across various spectral embedding methods.
    • A loss function is defined, clarifying the objective of traditional spectral embedding algorithms.
    • Experimental results confirm the effectiveness of the out-of-sample extension, showing good generalization performance.

    Conclusions:

    • Spectral embedding methods and KPCA are unified under a more general learning framework.
    • The proposed out-of-sample extension using the Nyström formula provides a robust way to generalize spectral embedding techniques.
    • The theoretical analysis clarifies the learning objective and generalization capabilities of these important algorithms.