Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Experiment Videos

Nonlinear feedforward networks with stochastic outputs: infomax implies redundancy reduction

J P Nadal1, N Brunel, N Parga

  • 1Laboratoire de Physique Statistique, Ecole Normale Supérieure, Paris, France.

Network (Bristol, England)
|December 23, 1998
PubMed
Summary

Maximizing mutual information in feedforward neural networks achieves full redundancy reduction. This holds when inputs are invertible mixtures of independent components and outputs are stochastic, extending previous deterministic findings.

Related Concept Videos

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

The AI Innerve Algorithm: Automatic Reproducible Reading for Intraepidermal Nerve Fiber Density Analysis, a New Tool for Small-Fiber Neuropathy Diagnosis.

European journal of neurology·2026
Same author

Emergence of Irregular Activity in Networks of Strongly Coupled Conductance-Based Neurons.

Physical review. X·2022
Same author

Astrocytes: Orchestrating synaptic plasticity?

Neuroscience·2015
Same author

Insights into Populus XIP aquaporins: evolutionary expansion, protein functionality, and environmental regulation.

Journal of experimental botany·2012
Same author

Effects of neuromodulation in a cortical network model of object working memory dominated by recurrent inhibition.

Journal of computational neuroscience·2001
Same author

Effects of synaptic noise and filtering on the frequency response of spiking neurons.

Physical review letters·2001

Area of Science:

  • Computational Neuroscience
  • Machine Learning Theory

Background:

  • Feedforward neural networks are key in AI and neuroscience.
  • Information theory principles, like mutual information, are used to understand neural computation.
  • Previous work (Nadal & Parga, 1994) explored redundancy reduction with deterministic outputs.

Purpose of the Study:

  • To prove that maximizing mutual information leads to full redundancy reduction in feedforward neural networks.
  • To extend previous findings to cases with stochastic neuron outputs.

Main Methods:

  • Mathematical proof based on information theory.
  • Analysis of mutual information maximization between network input and output.
  • Consideration of specific conditions on input signals and output neuron activities.

Related Experiment Videos

Main Results:

  • Demonstrated that maximizing mutual information guarantees full redundancy reduction under stated conditions.
  • Showcased that this principle applies even when output neuron activities are stochastic variables.
  • Extended the applicability of redundancy reduction principles to more complex neural network models.

Conclusions:

  • Maximizing mutual information is a sufficient condition for full redundancy reduction in feedforward neural networks.
  • The findings generalize previous results by including stochastic outputs, offering broader insights into neural information processing.