Related Experiment Video
Updated: Jan 10, 2026
11:15
Applications of EEG Neuroimaging Data: Event-related Potentials, Spectral Power, and Multiscale Entropy
Published on: June 27, 2013
34.3K
Information Entropy of Biometric Data in a Recurrent Neural Network with Low Connectivity.
David Dominguez-Carreta1, Mario González-Rodríguez2, Francisco B Rodriguez1
1Grupo de Neurocomputación Biológica, Dpto. de Ingeniería Informática, Escuela Politécnica Superior, Universidad Autónoma de Madrid, 28049 Madrid, Spain.
Entropy (Basel, Switzerland)
|November 26, 2025
Summary
This study reveals how low connectivity in random recurrent neural networks impacts storage capacity and information content. Network properties like connectivity and noise are crucial for performance in pattern retrieval tasks.
Area of Science:
- Computational neuroscience
- Machine learning theory
Background:
- Recurrent neural networks (RNNs) are powerful computational models.
- Understanding the storage capacity of RNNs with limited connectivity is crucial for efficient information processing.
Purpose of the Study:
- To investigate the storage capacity and maximal information content of random RNNs with low connectivity.
- To analyze the influence of network properties (connectivity, synaptic noise) on performance and information retention.
Main Methods:
- Theoretical analysis of network properties.
- Embedding specific patterns using Hebbian learning.
- Evaluating information retention via entropy measures.
- Complementing analysis with extensive simulations.
Main Results:
- Characterized storage capacity and information content in low-connectivity RNNs.
- Demonstrated the impact of connectivity and synaptic noise on network performance.
- Validated findings by comparing with biometric pattern retrieval (retinal maps, fingerprints).
Conclusions:
- Finite-connectivity RNNs have specific limitations and capabilities for pattern retrieval.
- Insights gained are applicable to understanding complex, structured pattern retrieval in neural networks.
Related Concept Videos
01:22Entropy within the Cell
12.6K
A living cell's primary tasks of obtaining, transforming, and using energy to do work may seem simple. However, the second law of thermodynamics explains why these tasks are harder than they appear. None of the energy transfers in the universe are completely efficient. In every energy transfer, some amount of energy is lost in a form that is unusable. In most cases, this form is heat energy. Thermodynamically, heat energy is defined as the energy transferred from one system to another that...
12.6K
01:10Entropy Change in Reversible Processes
3.2K
In the Carnot engine, which achieves the maximum efficiency between two reservoirs of fixed temperatures, the total change in entropy is zero. The observation can be generalized by considering any reversible cyclic process consisting of many Carnot cycles. Thus, it can be stated that the total entropy change of any ideal reversible cycle is zero.
The statement can be further generalized to prove that entropy is a state function. Take a cyclic process between any two points on a p-V diagram.
The statement can be further generalized to prove that entropy is a state function. Take a cyclic process between any two points on a p-V diagram.
3.2K
01:25Neural Circuits
2.6K
Neural circuits and neuronal pools are two of the main structures found in the nervous system. Neural circuits are networks of neurons that work together to carry out a specific task or process. They consist of interconnected neurons and glial cells, which provide structural and metabolic support.
Neuronal pools are collections of nerve cells with similar functions and interact through chemical and electrical signals. These pools include both interneurons (the central neural circuit nodes that...
Neuronal pools are collections of nerve cells with similar functions and interact through chemical and electrical signals. These pools include both interneurons (the central neural circuit nodes that...
2.6K
01:24Sequence Networks of Rotating Machines
473
A Y-connected synchronous generator, grounded through a neutral impedance, is designed to produce balanced internal phase voltages with only positive-sequence components. The generator's sequence networks include a source voltage that is exclusively in the positive-sequence network. The sequence components of line-to-ground voltages at the generator terminals illustrate this configuration.
Zero-sequence current induces a voltage drop across the generator's neutral impedance and other...
Zero-sequence current induces a voltage drop across the generator's neutral impedance and other...
473
01:20Entropy and the Second Law of Thermodynamics
4.7K
The second law of thermodynamics can be stated quantitatively using the concept of entropy. Entropy is the measure of disorder of the system.
The relation between entropy and disorder can be illustrated with the example of the phase change of ice to water. In ice, the molecules are located at specific sites giving a solid state, whereas, in a liquid form, these molecules are much freer to move. The molecular arrangement has therefore become more randomized. Although the change in average...
The relation between entropy and disorder can be illustrated with the example of the phase change of ice to water. In ice, the molecules are located at specific sites giving a solid state, whereas, in a liquid form, these molecules are much freer to move. The molecular arrangement has therefore become more randomized. Although the change in average...
4.7K
02:26Protein Networks
4.5K
An organism can have thousands of different proteins, and these proteins must cooperate to ensure the health of an organism. Proteins bind to other proteins and form complexes to carry out their functions. Many proteins interact with multiple other proteins creating a complex network of protein interactions.
These interactions can be represented through maps depicting protein-protein interaction networks, represented as nodes and edges. Nodes are circles that are representative of a protein,...
These interactions can be represented through maps depicting protein-protein interaction networks, represented as nodes and edges. Nodes are circles that are representative of a protein,...
4.5K