Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Experiment Videos

Unsupervised feature learning from finite data by message passing: Discontinuous versus continuous phase transition.

Haiping Huang1, Taro Toyoizumi1

  • 1RIKEN Brain Science Institute, Wako-shi, Saitama 351-0198, Japan.

Physical Review. E
|January 14, 2017
PubMed
Summary

Unsupervised learning, crucial for deep networks, was studied using a restricted Boltzmann machine. Researchers developed a message-passing algorithm revealing learning speed depends on feature strength and data size, showing an easy-hard-easy learning phenomenon.

Related Concept Videos

Linear Approximation in Frequency Domain01:26

Linear Approximation in Frequency Domain

412
Linear systems are characterized by two main properties: superposition and homogeneity. Superposition allows the response to multiple inputs to be the sum of the responses to each individual input. Homogeneity ensures that scaling an input by a scalar results in the response being scaled by the same scalar.
In contrast, nonlinear systems do not inherently possess these properties. However, for small deviations around an operating point, a nonlinear system can often be approximated as linear....
412
Classification of Signals01:30

Classification of Signals

1.5K
In signal processing, signals are classified based on various characteristics: continuous-time versus discrete-time, periodic versus aperiodic, analog versus digital, and causal versus noncausal. Each category highlights distinct properties crucial for understanding and manipulating signals.
A continuous-time signal holds a value at every instant in time, representing information seamlessly. In contrast, a discrete-time signal holds values only at specific moments, often denoted as x(n), where...
1.5K

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Single-cell transcriptomic analysis reveals dynamic changes in the microenvironment of nasopharyngeal carcinoma during metastatic progression.

Computational biology and chemistry·2026
Same author

Does the Apple airpods pro 2 hearing aid feature meet prescribed targets for standardized audiograms?

International journal of audiology·2026
Same author

Longitudinal Association Between Possible Sarcopenia and Stroke Under the AWGS 2025 Criteria: A Nationwide Prospective Cohort Study With a 9-Year Follow-Up.

Geriatrics & gerontology international·2026
Same author

How Speaker Configuration of Recorded Media Affects Subjective Ratings for Listeners With and Without Hearing Loss.

Journal of speech, language, and hearing research : JSLHR·2026
Same author

HSP47 inhibitor Col003 attenuates thromboinflammation after cerebral ischemia-reperfusion by suppressing GPVI-mediated CD84 shedding in platelets.

Molecular immunology·2026
Same author

Beyond variance: sensitivity-based dimensions in brain networks underlie individual differences in cognitive ability.

ArXiv·2026

Area of Science:

  • Artificial Intelligence
  • Machine Learning
  • Computational Neuroscience

Background:

  • Unsupervised learning extracts features from unlabeled data, essential for pretraining deep networks.
  • Understanding unsupervised learning from finite data is fundamental for advancing AI.
  • Restricted Boltzmann Machines (RBMs) are key models for studying unsupervised feature learning.

Purpose of the Study:

  • To analyze unsupervised learning from finite data using a simplified RBM with one hidden neuron.
  • To develop and evaluate an efficient message-passing algorithm for feature inference and entropy estimation.
  • To investigate the impact of data size and feature salience on learning dynamics and phase transitions.

Main Methods:

  • Theoretical analysis of unsupervised learning in a single-hidden-neuron RBM.

Related Experiment Videos

  • Development of a message-passing algorithm for inferring hidden features and estimating feature entropy.
  • Comparison with an approximate Hopfield model and validation on a handwritten digits dataset.

    • Main Results:

    • Learning efficiency is directly related to feature salience; strong features require less data.
    • Feature entropy decreases with data size, leading to an 'entropy crisis' and discontinuous phase transition.
    • The message-passing algorithm exhibits an 'easy-hard-easy' convergence time pattern with increasing data size.
    • An approximate Hopfield model reproduces most RBM properties but lacks the entropy crisis and shows continuous phase transition.

    Conclusions:

    • The study provides fundamental insights into unsupervised learning dynamics with finite data.
    • The findings highlight the role of feature salience and data size in learning efficiency and phase transitions.
    • The observed 'entropy crisis' and discontinuous transition in RBMs differ from continuous transitions in Hopfield models, with implications for deep network training.