Related Experiment Video
Updated: Jun 5, 2026
07:34
Perceptual and Category Processing of the Uncanny Valley Hypothesis' Dimension of Human Likeness: Some Methodological Issues
Published on: June 3, 2013
Bounding the bias of contrastive divergence learning
1Institut für Neuroinformatik, Ruhr-Universität Bochum, 44780 Bochum, Germany asja.fischer@ini.rub.de.
Neural Computation
|December 18, 2010
Summary
We derived a new upper bound for the bias in k-step contrastive divergence (CD) for training restricted Boltzmann machines (RBMs). This bias depends on RBM parameters, size, and Gibbs sampling chain variations.
Area of Science:
- Machine Learning
- Artificial Intelligence
- Computational Neuroscience
Background:
- Restricted Boltzmann Machines (RBMs) are widely used for unsupervised learning.
- Training RBMs often involves optimization using k-step contrastive divergence (CD).
- k-step CD is known to be a biased estimator of the log-likelihood gradient.
Purpose of the Study:
- To derive a novel upper bound for the bias inherent in k-step contrastive divergence (CD).
- To analyze the factors influencing the magnitude of this bias in RBM training.
Main Methods:
- Theoretical derivation of a new upper bound for the bias in k-step CD.
- Analysis of the bias's dependence on RBM characteristics and Gibbs sampling dynamics.
Main Results:
- The derived upper bound quantifies the bias based on the number of steps (k), RBM size, and energy changes.
- Bias magnitude is influenced by RBM parameter values and the initial distribution of the Gibbs chain.
Conclusions:
- Understanding this bias is crucial for effective RBM training.
- The new bound provides insights into the limitations of k-step CD and potential areas for improvement.
Related Concept Videos
01:05Difference from Background: Limit of Detection
The limit of detection (LOD) is the smallest amount of analyte that can be distinguished from the background noise. The LOD value corresponds to the concentration at which the analyte signal is three times larger than the standard deviation of the blank signal. Below this value, the analyte signal cannot be differentiated from the background noise. It is calculated by dividing the calibration slope by 3 times the standard deviation of the blank signals.
The LOD indicates the presence or absence...
The LOD indicates the presence or absence...
01:22Bias
Bias refers to any tendency that prevents a question from being considered unprejudiced. In research, bias occurs when one outcome or answer is selected or encouraged over others in sampling or testing. Bias can occur during any research phase, including study design, data collection, analysis, and publication.
In statistics, a sampling bias is created when a sample is collected from a population, and some members of the population are not as likely to be chosen as others (remember, each member...
In statistics, a sampling bias is created when a sample is collected from a population, and some members of the population are not as likely to be chosen as others (remember, each member...
01:22Biasing of FET
Biasing a Junction Field Effect Transistor (JFET) is crucial for setting operational parameters and ensuring efficient functioning in electronic circuits. JFETs are characterized by using a single carrier type in N-channel or P-channel configurations, where the channel is surrounded by PN junctions. These junctions are central to the device's ability to control current flow.
In an N-channel JFET, the structure consists of N-type material forming the channel on a P-type substrate, with the gate...
In an N-channel JFET, the structure consists of N-type material forming the channel on a P-type substrate, with the gate...
01:16Biasing of P-N Junction
The operation of a p-n junction diode involves various biasing conditions, including forward bias, reverse bias, and equilibrium.
In equilibrium, no external voltage is applied across the p-n junction. The depletion region is formed at the junction interface due to the diffusion of carriers, which leaves behind charged dopants, acceptors on the p-side, and donors on the n-side. These immobile charges create an electric field that prevents further diffusion of carriers. The related energy band...
In equilibrium, no external voltage is applied across the p-n junction. The depletion region is formed at the junction interface due to the diffusion of carriers, which leaves behind charged dopants, acceptors on the p-side, and donors on the n-side. These immobile charges create an electric field that prevents further diffusion of carriers. The related energy band...
01:17Correspondence Bias
Correspondence bias, also referred to as the fundamental attribution error, describes the tendency to attribute another person’s behavior to internal characteristics rather than situational influences. This cognitive bias leads individuals to overlook external factors that may be influencing actions, thereby fostering potentially inaccurate assessments of others’ intentions and dispositions.Empirical Evidence for Correspondence BiasResearch has consistently demonstrated the prevalence of...
01:26Causes of Similarity-Dissimilarity Effect
The similarity-dissimilarity effect, a fundamental concept in social psychology, explains how interpersonal similarities and differences influence attraction and social interactions. This effect is supported by three key psychological perspectives: balance theory, social comparison theory, and consensual validation.Balance Theory and Cognitive ConsistencyBalance theory, developed by Fritz Heider, posits that individuals seek cognitive consistency in their relationships. When two people share...