Related Experiment Video
Updated: May 16, 2026
08:27
Image Recognition and Parameter Analysis of Concrete Vibration State Based on Support Vector Machine
Published on: January 5, 2024
Enhanced gradient for training restricted Boltzmann machines
Kyunghyun Cho1, Tapani Raiko, Alexander Ilin
1Department of Information and Computer Science, Aalto University School of Science, Espoo, Uusimaa 02150, Finland. kyunghyun.cho@aalto.fi
Neural Computation
|November 15, 2012
Summary
Training Restricted Boltzmann machines (RBMs) is challenging due to sensitivity to hyperparameters and data representation. This study introduces an enhanced gradient invariant to bit-flipping, enabling more stable RBM training.
Area of Science:
- Machine Learning
- Deep Learning
- Artificial Intelligence
Background:
- Restricted Boltzmann Machines (RBMs) are foundational for deep learning.
- Traditional RBM training is laborious and sensitive to metaparameters and data representation.
- Existing learning rules lack invariance to bit-flipping transformations, leading to training instability.
Purpose of the Study:
- To develop a novel gradient for RBM training.
- To enhance training stability and robustness.
Main Methods:
- Derivation of an enhanced gradient invariant to bit-flipping transformations.
- Experimental validation of the enhanced gradient's performance.
Main Results:
- The enhanced gradient ensures invariance to bit-flipping.
- More stable RBM training observed with both fixed and adaptive learning rates.
- Reduced sensitivity to metaparameter tuning.
Conclusions:
- The proposed enhanced gradient significantly improves RBM training stability.
- This method offers a more robust approach to deep learning model development.
- Further research can explore its application in larger deep networks.
Related Concept Videos
01:20Maxwell-Boltzmann Distribution: Problem Solving
Individual molecules in a gas move in random directions, but a gas containing numerous molecules has a predictable distribution of molecular speeds, which is known as the Maxwell-Boltzmann distribution, f(v).
This distribution function f(v) is defined by saying that the expected number N (v1,v2) of particles with speeds between v1 and v2 is given by
This distribution function f(v) is defined by saying that the expected number N (v1,v2) of particles with speeds between v1 and v2 is given by
01:14Gradient and Del Operator
In mathematics and physics, the gradient and del operator are fundamental concepts used to describe the behavior of functions and fields in space. The gradient is a mathematical operator that gives both the magnitude and direction of the maximum spatial rate of change. Consider a person standing on a mountain. The slope of the mountain at any given point is not defined unless it is quantified in a particular direction. For this reason, a "directional derivative" is defined, which is a vector...