Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Experiment Videos

The "wake-sleep" algorithm for unsupervised neural networks

G E Hinton1, P Dayan, B J Frey

  • 1Department of Computer Science, University of Toronto, Ontario, Canada.

Science (New York, N.Y.)
|May 26, 1995
PubMed
Summary
This summary is machine-generated.

Related Concept Videos

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Knowing me, knowing you: Interpersonal similarity improves predictive accuracy and reduces attributions of harmful intent.

Cognition·2022
Same author

Disrupted habenula function in major depression.

Molecular psychiatry·2016
Same author

The specificity of Pavlovian regulation is associated with recovery from depression.

Psychological medicine·2016
Same author

Memory, modelling and Marr: a commentary on Marr (1971) 'Simple memory: a theory of archicortex'.

Philosophical transactions of the Royal Society of London. Series B, Biological sciences·2015
Same author

How receptor diffusion influences gradient sensing.

Journal of the Royal Society, Interface·2015
Same author

The influence of receptor positioning on chemotactic information.

Journal of theoretical biology·2014
Same journal

A native sulfur deposit in Gale crater, Mars.

Science (New York, N.Y.)·2026
Same journal

Coordinated demise of harmful algal blooms.

Science (New York, N.Y.)·2026
Same journal

Genetic effects put into context.

Science (New York, N.Y.)·2026
Same journal

Bacteria share proteins to survive antibiotics.

Science (New York, N.Y.)·2026
Same journal

Impacts shaped Earth's first continents.

Science (New York, N.Y.)·2026
Same journal

Erratum for the Report "Covalently bonded single-molecule junctions with stable and reversible photoswitched conductivity" by C. Jia <i>et al</i>.

Science (New York, N.Y.)·2026
See all related articles

This study introduces an unsupervised learning algorithm for stochastic neural networks. It uses distinct "wake" and "sleep" phases to train both recognition and generative connections for improved data representation and reconstruction.

Area of Science:

  • Computational Neuroscience
  • Machine Learning
  • Artificial Intelligence

Background:

  • Artificial neural networks often require large labeled datasets for training.
  • Unsupervised learning methods aim to learn representations from unlabeled data.
  • Hierarchical processing in biological brains inspires layered network architectures.

Purpose of the Study:

  • To describe a novel unsupervised learning algorithm for multilayer stochastic neural networks.
  • To enable networks to learn representations and reconstruct data without explicit labels.
  • To investigate the interplay between recognition and generative processes in neural computation.

Main Methods:

  • Development of an unsupervised learning algorithm for a multilayer network of stochastic neurons.

Related Experiment Videos

  • Implementation of bottom-up 'recognition' connections for input-to-representation mapping.
  • Implementation of top-down 'generative' connections for representation-to-reconstruction mapping.
  • Two distinct phases: 'wake' phase (recognition-driven) and 'sleep' phase (generative-driven).
  • Adaptation of connection weights to improve probability of correct activity vector reconstruction and production.

    • Main Results:

    • The algorithm successfully trains both recognition and generative connections in a multilayer network.
    • The 'wake' phase optimizes generative connections for accurate bottom-up reconstruction.
    • The 'sleep' phase optimizes recognition connections for accurate top-down production.
    • This dual-phase approach allows for unsupervised learning of hierarchical representations.

    Conclusions:

    • The proposed unsupervised learning algorithm effectively trains stochastic neural networks.
    • The 'wake-sleep' mechanism facilitates learning of both feature extraction and data generation.
    • This method offers a promising approach for developing more biologically plausible and efficient AI systems.