Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

Propagation of Action Potentials01:23

Propagation of Action Potentials

8.4K
The propagation of an action potential refers to the process by which a nerve impulse, or "action potential," travels along a neuron.
Neurons (nerve cells) have a resting membrane potential, with a slightly negative charge inside compared to outside. This is maintained by ion channels, such as sodium (Na+) and potassium (K+) channels, which control the flow of ions. When a stimulus, like a touch or a signal from another neuron, triggers the neuron, sodium channels open, allowing sodium ions to...
8.4K
Improving Translational Accuracy02:07

Improving Translational Accuracy

13.8K
Base complementarity between the three base pairs of mRNA codon and the tRNA anticodon is not a failsafe mechanism. Inaccuracies can range from a single mismatch to no correct base pairing at all. The free energy difference between the correct and nearly correct base pairs can be as small as 3 kcal/ mol. With complementarity being the only proofreading step, the estimated error frequency would be one wrong amino acid in every 100 amino acids incorporated. However, error frequencies observed in...
13.8K
Improving Translational Accuracy02:07

Improving Translational Accuracy

3.4K
3.4K
Neural Circuits01:25

Neural Circuits

2.4K
Neural circuits and neuronal pools are two of the main structures found in the nervous system. Neural circuits are networks of neurons that work together to carry out a specific task or process. They consist of interconnected neurons and glial cells, which provide structural and metabolic support.
Neuronal pools are collections of nerve cells with similar functions and interact through chemical and electrical signals. These pools include both interneurons (the central neural circuit nodes that...
2.4K
Observational Learning01:12

Observational Learning

722
Albert Bandura's observational learning, also known as imitation or modeling, occurs when a person observes and imitates another's behavior. It is a quicker process than operant conditioning. A well-known example is the Bobo doll study, where children who saw an adult acting aggressively towards the doll were more likely to act aggressively when left alone, compared to those who observed a nonaggressive adult. Many psychologists view observational learning as a form of latent learning...
722
Long-term Potentiation01:25

Long-term Potentiation

3.2K
Long-term potentiation, or LTP, is one of the ways by which synaptic plasticity—changes in the strength of chemical synapses—can occur in the brain. LTP is the process of synaptic strengthening that occurs over time between pre and postsynaptic neuronal connections. The synaptic strengthening of LTP works in opposition to the synaptic weakening of long-term depression (LTD) and together are the main mechanisms that underlie learning and memory.
Hebbian LTP
LTP can occur when...
3.2K

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Strategies to Improve the Impact of Artificial Intelligence on Health Equity: Scoping Review.

JMIR AI·2024
Same author

Noise can speed Markov chain Monte Carlo estimation and quantum annealing.

Physical review. E·2019
Same author

Noise-boosted bidirectional backpropagation and adversarial learning.

Neural networks : the official journal of the International Neural Network Society·2019
Same author

An open challenge to advance probabilistic forecasting for dengue epidemics.

Proceedings of the National Academy of Sciences of the United States of America·2019
Same author

Modeling multiple time series annotations as noisy distortions of the ground truth: An Expectation-Maximization approach.

IEEE transactions on affective computing·2018
Same author

Detecting paralinguistic events in audio stream using context in features and probabilistic decisions.

Computer speech & language·2017

Related Experiment Video

Updated: Dec 17, 2025

Decoding Natural Behavior from Neuroethological Embedding
08:00

Decoding Natural Behavior from Neuroethological Embedding

Published on: October 3, 2025

444

Noise can speed backpropagation learning and deep bidirectional pretraining.

Bart Kosko1, Kartik Audhkhasi2, Osonde Osoba3

  • 1Department of Electrical and Computer Engineering, Signal and Image Processing Institute, University of Southern California, Los Angeles, CA 90089-2564, USA.

Neural Networks : the Official Journal of the International Neural Network Society
|June 30, 2020
PubMed
Summary

Adding specific noise accelerates neural network training and pretraining. This beneficial noise, making signals more probable, enhances convergence and classification accuracy in multilayer networks and bidirectional associative memories.

Keywords:
BackpropagationBidirectional associative memoryContrastive divergenceExpectation–Maximization algorithmNoise benefitStochastic resonance

Related Experiment Videos

Last Updated: Dec 17, 2025

Decoding Natural Behavior from Neuroethological Embedding
08:00

Decoding Natural Behavior from Neuroethological Embedding

Published on: October 3, 2025

444

Area of Science:

  • Artificial Intelligence
  • Machine Learning
  • Computational Neuroscience

Background:

  • The backpropagation algorithm is a cornerstone of training multilayer neural networks.
  • Iterative maximum likelihood estimation is crucial for model optimization.
  • The Expectation-Maximization (EM) algorithm offers a framework for such estimation.

Purpose of the Study:

  • To demonstrate that backpropagation is a special case of the generalized Expectation-Maximization (EM) algorithm.
  • To investigate the impact of carefully chosen noise on the convergence speed of EM and backpropagation.
  • To explore the potential of noise injection for improving neural network training, pretraining, and classification accuracy.

Main Methods:

  • Established backpropagation as a specific instance of the generalized EM algorithm.
  • Applied recent findings on noise-assisted convergence in EM to backpropagation.
  • Injected probabilistically beneficial noise into hidden and visible neurons and parameters of neural networks.
  • Utilized MNIST digit classification simulations to demonstrate noise benefits.
  • Extended the analysis to deep bidirectional pretraining of neural-network bidirectional associative memories (BAMs) and restricted Boltzmann machines.

Main Results:

  • Injecting noise that increases signal probability significantly speeds up the average convergence of backpropagation for training and pretraining.
  • This beneficial noise improves classification accuracy and outperforms random noise addition.
  • The noise-benefit region's geometry is dependent on the neuron's probability structure, forming a hyperplane for classification and a hypersphere for regression.
  • Noise-induced speed-up was proven applicable to deep bidirectional pretraining of BAMs and restricted Boltzmann machines.
  • Learning with contrastive divergence was shown to reduce to generalized EM for energy-based networks.

Conclusions:

  • Carefully chosen noise can accelerate neural network training and pretraining by enhancing convergence and accuracy.
  • The generalized EM framework provides a theoretical basis for understanding noise benefits in backpropagation.
  • Noise injection offers a promising technique for optimizing deep learning models, including BAMs and restricted Boltzmann machines.