Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Experiment Videos

Improving dimensionality reduction with spectral gradient descent.

Roland Memisevic1, Geoffrey Hinton

  • 1Department of Computer Science, University of Toronto, Toronto, Ont., Canada. roland@cs.toronto.edu

Neural Networks : the Official Journal of the International Neural Network Society
|August 23, 2005
PubMed
Summary
This summary is machine-generated.

Related Concept Videos

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Managing extreme AI risks amid rapid progress.

Science (New York, N.Y.)·2024
Same author

Robust and data-efficient generalization of self-supervised machine learning for diagnostic imaging.

Nature biomedical engineering·2023
Same author

How to Represent Part-Whole Hierarchies in a Neural Network.

Neural computation·2022
Same author

Backpropagation and the brain.

Nature reviews. Neuroscience·2020
Same author

Deep Learning-A Technology With the Potential to Transform Health Care.

JAMA·2018
Same author

The Potential Energy of an Autoencoder.

IEEE transactions on pattern analysis and machine intelligence·2015
Same journal

Exploiting audio-visual modalities in videos: Object detection via multi-stage bilateral coupling network.

Neural networks : the official journal of the International Neural Network Society·2026
Same journal

Reliability-aware modality completion with cross-modal distillation for federated learning with missing modalities.

Neural networks : the official journal of the International Neural Network Society·2026
Same journal

IGFD-Net: Illumination-guided frequency decoupling for polarization image fusion.

Neural networks : the official journal of the International Neural Network Society·2026
Same journal

Multiple-Strategies dung beetle optimizer and its applications in engineering optimization and bankruptcy prediction.

Neural networks : the official journal of the International Neural Network Society·2026
Same journal

Aggregating global-scale pixel-wise forgery cues within a graph.

Neural networks : the official journal of the International Neural Network Society·2026
Same journal

Finite-Time intermittent control for secure synchronization of Neutral-Type stochastic delayed neural networks under aperiodic DoS attacks.

Neural networks : the official journal of the International Neural Network Society·2026
See all related articles

Spectral gradient descent enhances iterative dimensionality reduction by using data affinity matrix eigenvalues. This method accelerates optimization and improves results in finding better local minima.

Area of Science:

  • Machine Learning
  • Data Science
  • Numerical Optimization

Background:

  • Iterative dimensionality reduction techniques are crucial for data analysis.
  • Gradient-based optimization methods can be slow and may converge to suboptimal solutions.
  • Existing methods lack efficient ways to leverage matrix properties for faster convergence.

Purpose of the Study:

  • To introduce Spectral Gradient Descent (SGD) for improving iterative dimensionality reduction.
  • To accelerate the optimization process in dimensionality reduction.
  • To enhance the ability of methods to find better local minima.

Main Methods:

  • Developed Spectral Gradient Descent (SGD) by incorporating leading eigenvalues of a data affinity matrix.
  • Modified gradient-based optimization steps using spectral information.

Related Experiment Videos

  • Interpreted the approach using the power method for symmetric matrix eigenvalue computation.

    • Main Results:

    • Demonstrated that SGD speeds up the optimization process.
    • Showed that SGD helps dimensionality reduction methods find improved local minima.
    • Verified the approach's effectiveness through experimental validation.

    Conclusions:

    • Spectral Gradient Descent offers a significant improvement over standard iterative dimensionality reduction techniques.
    • The method provides a novel way to accelerate convergence and enhance solution quality.
    • The findings suggest broad applicability in various data analysis and machine learning tasks.