Related Experiment Videos
Deterministic annealing techniques for a discrete-time neural-network updating in a block-sequential mode
1Olympus Optical Co., Ltd., 2-3 Kuboyama-cho, Hachioji, Tokyo 192, Japan.
IEEE Transactions on Neural Networks
|February 7, 2008
Summary
This study derives a stability criterion for discrete-time neural networks, introducing deterministic annealing techniques. These methods ensure stable network updates and achieve near-optimal solutions for various updating modes.
Area of Science:
- Artificial Intelligence
- Computational Neuroscience
- Machine Learning
Background:
- Discrete-time neural networks require stable updating mechanisms for reliable performance.
- Block-sequential updating modes present unique challenges for maintaining stability.
- Deterministic annealing offers a potential approach to optimize neural network training.
Purpose of the Study:
- To derive a global stability criterion for key parameters in discrete-time neural network block-sequential updating.
- To investigate two deterministic annealing techniques based on this stability criterion.
- To demonstrate the effectiveness of these techniques for achieving stable and near-optimal solutions.
Main Methods:
- Derivation of a global stability criterion for two constituent parameters.
- Implementation of deterministic annealing by gradually reducing membrane potential decay rate.
- Implementation of deterministic annealing by gradually increasing neuron gain.
- Analysis of stability for parallel, partial-parallel, and sequential updating modes.
Main Results:
- A global stability criterion for discrete-time neural network parameters was successfully derived.
- Two deterministic annealing techniques were studied, incorporating the derived stability condition.
- Stable network updates without sustained oscillations were achieved through controlled parameter adjustments.
- Near-optimal solutions were obtained across parallel, partial-parallel, and sequential updating schemes.
Conclusions:
- The derived stability criterion is crucial for stable discrete-time neural network operation.
- Deterministic annealing techniques effectively prevent oscillations and ensure convergence.
- Careful selection of parameters allows for optimal solutions in various updating scenarios.
Related Concept Videos
01:35Long-term Potentiation
Long-term potentiation, or LTP, is one of the ways by which synaptic plasticity—changes in the strength of chemical synapses—can occur in the brain. LTP is the process of synaptic strengthening that occurs over time between pre- and postsynaptic neuronal connections. The synaptic strengthening of LTP works in opposition to the synaptic weakening of long-term depression (LTD) and together are the main mechanisms that underlie learning and memory.
01:25Long-term Potentiation
Long-term potentiation, or LTP, is one of the ways by which synaptic plasticity—changes in the strength of chemical synapses—can occur in the brain. LTP is the process of synaptic strengthening that occurs over time between pre and postsynaptic neuronal connections. The synaptic strengthening of LTP works in opposition to the synaptic weakening of long-term depression (LTD) and together are the main mechanisms that underlie learning and memory.
Hebbian LTP
LTP can occur when presynaptic neurons...
Hebbian LTP
LTP can occur when presynaptic neurons...
01:29Mechanistic Models: Compartment Models in Algorithms for Numerical Problem Solving
Mechanistic models play a crucial role in algorithms for numerical problem-solving, particularly in nonlinear mixed effects modeling (NMEM). These models aim to minimize specific objective functions by evaluating various parameter estimates, leading to the development of systematic algorithms. In some cases, linearization techniques approximate the model using linear equations.
In individual population analyses, different algorithms are employed, such as Cauchy's method, which uses a...
In individual population analyses, different algorithms are employed, such as Cauchy's method, which uses a...
01:25Neural Circuits
Neural circuits and neuronal pools are two of the main structures found in the nervous system. Neural circuits are networks of neurons that work together to carry out a specific task or process. They consist of interconnected neurons and glial cells, which provide structural and metabolic support.
Neuronal pools are collections of nerve cells with similar functions and interact through chemical and electrical signals. These pools include both interneurons (the central neural circuit nodes that...
Neuronal pools are collections of nerve cells with similar functions and interact through chemical and electrical signals. These pools include both interneurons (the central neural circuit nodes that...
01:22Block Diagram Reduction
The process of deriving the transfer function of a control system often involves reducing its block diagram to a single block. This simplification can be achieved through a series of strategic operations, including relocating branch points and comparators. These operations preserve the overall function of the system while allowing for easier manipulation and combination of blocks.
The first step in this process is the identification and relocation of a branch point. A branch point, where a...
The first step in this process is the identification and relocation of a branch point. A branch point, where a...
01:25Elements of Block Diagrams
Block diagrams serve as a visual representation of the input-output relationships within a system. An illustrative example is a heating system, where the set temperature activates the furnace to warm the room to the desired level. Block diagrams are versatile, modeling linear systems through Laplace transform variables and nonlinear systems using time domain variables.
A block diagram typically includes essential elements such as comparators, blocks, and feedback loops. Each of these elements...
A block diagram typically includes essential elements such as comparators, blocks, and feedback loops. Each of these elements...