Search research articles

ABOUT JoVE

Overview Leadership Blog JoVE Help Center

AUTHORS

Publishing Process Editorial Board Scope & Policies Peer Review FAQ Submit

LIBRARIANS

Testimonials Subscriptions Access Resources Library Advisory Board FAQ

RESEARCH

JoVE Journal Methods Collections JoVE Encyclopedia of Experiments Archive

EDUCATION

JoVE Core JoVE Business JoVE Science Education JoVE Lab Manual Faculty Resource Center Faculty Site

Terms & Conditions of Use

Search research articles

Related Experiment Videos

Learning multiple layers of representation.

Geoffrey E Hinton¹

¹Department of Computer Science, University of Toronto, 10 King's College Road, Toronto, M5S 3G4, Canada. hinton@cs.toronto.edu

Trends in Cognitive Sciences

|October 9, 2007

Summary

This summary is machine-generated.

Related Concept Videos

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by

Same author

Adaptive Mixtures of Local Experts.

Neural computation·2019

Same author

Modeling natural images using gated MRFs.

IEEE transactions on pattern analysis and machine intelligence·2013

Same author

Machine learning for neuroscience.

Neural systems & circuits·2012

Same author

Comparing classification methods for longitudinal fMRI studies.

Neural computation·2010

Same author

Learning to represent spatial transformations with factored higher-order Boltzmann machines.

Neural computation·2010

Same author

Learning to represent visual input.

Philosophical transactions of the Royal Society of London. Series B, Biological sciences·2009

Same journal

Geographical psychology: Spatial variation in psychological phenomena and their consequences.

Trends in cognitive sciences·2026

Same journal

Multi-brain neurofeedback: what are we training for?

Trends in cognitive sciences·2026

Same journal

The developing vocal self.

Trends in cognitive sciences·2026

Same journal

Searching beyond decrements: Attentional guidance across the adult lifespan.

Trends in cognitive sciences·2026

Same journal

Looking into working memory through micro eye movements.

Trends in cognitive sciences·2026

Same journal

Timescapes of non-human experience.

Trends in cognitive sciences·2026

See all related articles

New neural network models overcome limitations of backpropagation by using top-down connections. This approach enables efficient learning of deep representations for complex tasks like speech perception.

Area of Science:

Computational neuroscience
Machine learning

Background:

The brain processes information through multiple representational layers.
Backpropagation, an early neural network learning algorithm, had limitations in deep networks and required labeled data.
Efficient learning in deep neural networks remains a significant challenge.

Purpose of the Study:

To present a novel approach for training deep neural networks that overcomes the limitations of backpropagation.
To enable neural networks to learn hierarchical representations effectively for complex sensory processing tasks.

Main Methods:

Utilizing multilayer neural networks with top-down connections.
Training networks to generate sensory data (generative models) instead of classifying it.
Employing a recent discovery for efficient layer-by-layer learning of nonlinear distributed representations.

Related Experiment Videos

Main Results:

Demonstrated a method to overcome backpropagation's limitations in deep networks.
Enabled efficient learning of multilayer generative models.
Facilitated the learning of nonlinear distributed representations in a hierarchical manner.

Conclusions:

Multilayer neural networks with top-down connections offer a viable alternative to backpropagation for deep learning.
Training generative models by generating sensory data is an effective strategy for learning complex representations.
Recent advancements simplify the learning of nonlinear distributed representations, paving the way for more powerful AI.