Search research articles

ABOUT JoVE

Overview Leadership Blog JoVE Help Center

AUTHORS

Publishing Process Editorial Board Scope & Policies Peer Review FAQ Submit

LIBRARIANS

Testimonials Subscriptions Access Resources Library Advisory Board FAQ

RESEARCH

JoVE Journal Methods Collections JoVE Encyclopedia of Experiments Archive

EDUCATION

JoVE Core JoVE Business JoVE Science Education JoVE Lab Manual Faculty Resource Center Faculty Site

Terms & Conditions of Use

Related Concept Videos

PI Controller: Design

PI Controller: Design

Proportional Integral (PI) controllers are a fundamental component in modern control systems, widely used to enhance performance and mitigate steady-state errors. They are particularly effective in applications such as automatic brightness adjustment on smartphones, where they excel at mitigating steady-state errors for step-function inputs. Unlike PD controllers, which require time-varying errors to function optimally, PI controllers leverage their integral component to address residual...

Survival Tree

Survival Tree

Survival trees are a non-parametric method used in survival analysis to model the relationship between a set of covariates and the time until an event of interest occurs, often referred to as the "time-to-event" or "survival time." This method is particularly useful when dealing with censored data, where the event has not occurred for some individuals by the end of the study period, or when the exact time of the event is unknown.
Building a Survival Tree
Constructing a...

Time-Domain Interpretation of PD Control

Time-Domain Interpretation of PD Control

Proportional-Derivative (PD) control is a widely used control method in various engineering systems to enhance stability and performance. In a system with only proportional control, common issues include high maximum overshoot and oscillation, observed in both the error signal and its rate of change. This behavior can be divided into three distinct phases: initial overshoot, subsequent undershoot, and gradual stabilization.
Consider the example of control of motor torque. Initially, a positive...

The Power Flow Problem and Solution

The Power Flow Problem and Solution

Power flow problem analysis is fundamental for determining real and reactive power flows in network components, such as transmission lines, transformers, and loads. The power system's single-line diagram provides data on the bus, transmission line, and transformer. Each bus k in the system is characterized by four key variables: voltage magnitude Vk, phase angle δk, real power Pk, and reactive power Qk. Two of these four variables are inputs, while the...

Propagation of Action Potentials

Propagation of Action Potentials

The propagation of an action potential refers to the process by which a nerve impulse, or "action potential," travels along a neuron.
Neurons (nerve cells) have a resting membrane potential, with a slightly negative charge inside compared to outside. This is maintained by ion channels, such as sodium (Na+) and potassium (K+) channels, which control the flow of ions. When a stimulus, like a touch or a signal from another neuron, triggers the neuron, sodium channels open, allowing sodium ions to...

Propagation of Uncertainty from Systematic Error

Propagation of Uncertainty from Systematic Error

The atomic mass of an element varies due to the relative ratio of its isotopes. A sample's relative proportion of oxygen isotopes influences its average atomic mass. For instance, if we were to measure the atomic mass of oxygen from a sample, the mass would be a weighted average of the isotopic masses of oxygen in that sample. Since a single sample is not likely to perfectly reflect the true atomic mass of oxygen for all the molecules of oxygen on Earth, the mass we obtain from this...

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by

Same author

Iterative Retrieval and Block Coding in Autoassociative and Heteroassociative Memory.

Neural computation·2019

Same author

Toward Self-Referential Autonomous Learning of Object and Situation Models.

Cognitive computation·2016

Same author

Structural Plasticity, Effectual Connectivity, and Memory in Cortex.

Frontiers in neuroanatomy·2016

Same author

Efficient Associative Computation with Discrete Synapses.

Neural computation·2015

Same author

Cognitive representations and cognitive processing of team-specific tactics in soccer.

PloS one·2015

Same author

The influence of reducing intermediate target constraints on grasp posture planning during a three-segment object manipulation task.

Experimental brain research·2014

Same journal

A Model-Free Reinforcement Learning Implementation of Decision Making Under Uncertainty by Sequential Sampling.

Neural computation·2026

Same journal

DROP: Distributional and Regular Optimism and Pessimism for Reinforcement Learning.

Neural computation·2026

Same journal

Hierarchical Active Inference Using Successor Representations.

Neural computation·2026

Same journal

W-Kernel and Its Principal Space for Frequentist Evaluation of Bayesian Estimators.

Neural computation·2026

Same journal

A Hidden Markov Model-Inspired Sequence Classification Method for Hyperdimensional Computing.

Neural computation·2026

Same journal

Sparse Graphical Modeling for Electrophysiological Phase-Based Connectivity Using Circular Statistics.

Neural computation·2026

See all related articles

Search research articles

Related Experiment Video

Updated: Oct 26, 2025

An Experimental Platform to Study the Closed-loop Performance of Brain-machine Interfaces

An Experimental Platform to Study the Closed-loop Performance of Brain-machine Interfaces

Published on: March 10, 2011

Power Function Error Initialization Can Improve Convergence of Backpropagation Learning in Neural Networks for

Andreas Knoblauch¹

¹Albstadt-Sigmaringen University, Albstadt 72458, Germany knoblauch@hs-albsig.de.

Neural Computation

|July 26, 2021

Summary

This summary is machine-generated.

A new error initialization method using power functions improves neural network training speed and convergence. This approach, generalizing cross-entropy loss, offers better gradient flow and avoids vanishing gradients in deep and recurrent networks.

More Related Videos

Deep Neural Networks for Image-Based Dietary Assessment

Deep Neural Networks for Image-Based Dietary Assessment

Published on: March 13, 2021

Related Experiment Videos

Last Updated: Oct 26, 2025

An Experimental Platform to Study the Closed-loop Performance of Brain-machine Interfaces

An Experimental Platform to Study the Closed-loop Performance of Brain-machine Interfaces

Published on: March 10, 2011

Deep Neural Networks for Image-Based Dietary Assessment

Deep Neural Networks for Image-Based Dietary Assessment

Published on: March 13, 2021

Area of Science:

Machine Learning
Artificial Intelligence
Deep Learning

Background:

Supervised learning in neural networks involves minimizing a loss function to align model predictions with target values.
Backpropagation relies on error signals initialized at the output layer, typically using the difference between predictions and targets.
Common loss functions like cross-entropy and sum of squared errors use specific error initialization strategies.

Purpose of the Study:

To evaluate a generalized error initialization method for neural network backpropagation using power functions |yn- tn|q.
To introduce a new family of loss functions that generalize cross-entropy.
To investigate the impact of this method on learning speed and convergence, particularly in deep and recurrent neural networks.

Main Methods:

Implementing and testing a novel error initialization strategy based on power functions |yn- tn|q for q>0.
Comparing the performance of the new loss functions against traditional ones like cross-entropy.
Conducting experiments across various learning tasks, including those involving deep and recurrent neural networks.

Main Results:

A proper choice of the exponent q in the power function significantly enhances the speed and convergence of backpropagation learning.
The new loss functions demonstrate improved fitting to the distribution of output layer error signals, leading to more efficient likelihood maximization.
The proposed error initialization procedure often yields a better gradient-to-loss ratio, mitigating issues like vanishing gradients.

Conclusions:

The generalized error initialization using power functions offers a promising alternative to standard methods for training neural networks.
This approach can lead to faster and more stable learning, especially in complex architectures like deep and recurrent neural networks.
The findings suggest that optimizing error initialization is crucial for maximizing model likelihood and navigating challenging loss landscapes.