Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Experiment Videos

Contrastive divergence in gaussian diffusions.

Javier R Movellan1

  • 1Institute for Neural Computation, University of California San Diego, La Jolla, CA 92093-0515, USA. movellan@mplab.ucsd.edu

Neural Computation
|April 29, 2008
PubMed
Summary
This summary is machine-generated.

Related Concept Videos

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Spike count distributions, factorizability, and contextual effects in area V1.

Neurocomputing·2010
Same author

Detecting contingencies: an infomax approach.

Neural networks : the official journal of the International Neural Network Society·2010
Same author

Tracking motion, deformation, and texture using conditionally gaussian processes.

IEEE transactions on pattern analysis and machine intelligence·2010
Same author

Socialization between toddlers and robots at an early childhood education center.

Proceedings of the National Academy of Sciences of the United States of America·2007
Same author

A Monte Carlo EM approach for partially observable diffusion processes: theory and applications to neural networks.

Neural computation·2002
Same journal

A Model-Free Reinforcement Learning Implementation of Decision Making Under Uncertainty by Sequential Sampling.

Neural computation·2026
Same journal

DROP: Distributional and Regular Optimism and Pessimism for Reinforcement Learning.

Neural computation·2026
Same journal

Hierarchical Active Inference Using Successor Representations.

Neural computation·2026
Same journal

W-Kernel and Its Principal Space for Frequentist Evaluation of Bayesian Estimators.

Neural computation·2026
Same journal

A Hidden Markov Model-Inspired Sequence Classification Method for Hyperdimensional Computing.

Neural computation·2026
Same journal

Sparse Graphical Modeling for Electrophysiological Phase-Based Connectivity Using Circular Statistics.

Neural computation·2026
See all related articles

Contrastive divergence (CD) learning in continuous-time linear stochastic neural networks can fail. CD may diverge or find incorrect solutions unless the network structure matches specific distribution moments, highlighting the need for theoretical improvements.

Area of Science:

  • Machine Learning
  • Computational Neuroscience
  • Artificial Intelligence

Background:

  • Contrastive divergence (CD) is a key learning algorithm for stochastic neural networks.
  • Continuous-time linear stochastic neural networks offer a tractable model for theoretical analysis.
  • Understanding CD's convergence properties is crucial for reliable AI model training.

Purpose of the Study:

  • To analyze the behavior of the contrastive divergence (CD) learning algorithm.
  • To investigate CD's convergence properties in continuous-time linear stochastic neural networks.
  • To identify conditions under which CD succeeds or fails.

Main Methods:

  • Mathematical analysis of the contrastive divergence (CD) algorithm.
  • Application of established techniques for analyzing continuous-time linear stochastic systems.

Related Experiment Videos

  • Examination of network structure's impact on learning algorithm convergence.

    • Main Results:

    • CD converges to maximum likelihood solutions only when network structure matches the first moments of the target distribution.
    • CD can converge to solutions significantly different from log-likelihood solutions.
    • CD may diverge entirely under certain network configurations.

    Conclusions:

    • The study reveals critical limitations of CD in specific neural network architectures.
    • Improved theoretical understanding is needed to predict and mitigate CD failures.
    • Findings suggest practical strategies for enhancing CD algorithm performance and reliability.