Search research articles

ABOUT JoVE

Overview Leadership Blog JoVE Help Center

AUTHORS

Publishing Process Editorial Board Scope & Policies Peer Review FAQ Submit

LIBRARIANS

Testimonials Subscriptions Access Resources Library Advisory Board FAQ

RESEARCH

JoVE Journal Methods Collections JoVE Encyclopedia of Experiments Archive

EDUCATION

JoVE Core JoVE Business JoVE Science Education JoVE Lab Manual Faculty Resource Center Faculty Site

Terms & Conditions of Use

Related Concept Videos

Significance of the Gradient Vector

Significance of the Gradient Vector

A surface defined by a function of two variables can be understood by examining how it changes along specific directions. When one variable is held constant, the surface reduces to a curve that reflects variation in the other variable. For example, fixing one variable and moving parallel to a coordinate axis produces a cross-sectional curve. The slope of this curve at a given point represents how the function changes in that particular direction, providing a measure of local steepness.By...

Gradient Vectors and Their Applications

Gradient Vectors and Their Applications

Every point on a topographical map corresponds to a particular elevation, so the landscape can be modeled as a surface whose height depends on horizontal position. From any given location, a hiker may face infinitely many directions, but only one direction produces the fastest possible increase in elevation. This unique route is called the direction of steepest ascent, and in multivariable calculus, it is represented by the gradient vector of the elevation function.The gradient vector points...

Newton’s Method

Newton’s Method

Newton’s Method is a powerful iterative technique for approximating the roots of real-valued, differentiable functions, particularly when analytical solutions are impractical. This approach is widely used in scientific computing, engineering, and finance, where equations may be too complex for traditional algebraic methods to handle. The method relies on an iterative process that refines an initial estimate using the function’s derivative to approach the true solution progressively.

Lagrange Multipliers: Two Constraints

Lagrange Multipliers: Two Constraints

The method of Lagrange multipliers with two constraints is used to optimize a function subject to two independent constraints. In many applications, the objective function represents a quantity to be maximized or minimized, such as cost, area, distance, or energy. The two constraints represent requirements that the solution must satisfy, such as fixed volume, limited resources, or prescribed dimensions.For a function of three variables, each constraint forms a surface in three-dimensional space.

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by

Same author

Vibrational and structural properties of tetramethyltin under pressure.

The Journal of chemical physics·2013

Same author

[Risk factors on the unintentional injuries among rural children aged 0-12 in Shaanxi province].

Zhonghua liu xing bing xue za zhi = Zhonghua liuxingbingxue zazhi·2013

Same author

Adcyap1r1 genotype, posttraumatic stress disorder, and depression among women exposed to childhood maltreatment.

Depression and anxiety·2013

Same author

Current status and challenge of Human Parasitology teaching in China.

Pathogens and global health·2012

Same author

Molecular characterization of heterogeneous mesenchymal stem cells with single-cell transcriptomes.

Biotechnology advances·2012

Same author

Surgical treatment of ossification of the ligamentum flavum associated with dural ossification in the thoracic spine.

Journal of clinical neuroscience : official journal of the Neurosurgical Society of Australasia·2012

Search research articles

Related Experiment Video

Updated: Jul 10, 2026

Deep Neural Networks for Image-Based Dietary Assessment

Deep Neural Networks for Image-Based Dietary Assessment

Published on: March 13, 2021

Training pi-sigma network by online gradient algorithm with penalty for small weight update.

Yan Xiong¹, Wei Wu, Xidai Kang

¹Department of Applied Mathematics, Dalian University of Technology, Dalian 116024, People's Republic of China. xiongyan888@sohu.com

Neural Computation

|November 1, 2007

Summary

Training pi-sigma networks with online gradient algorithms is slow. Introducing an adaptive penalty term to the error function accelerates convergence by increasing weight update increments, enhancing learning speed.

Related Experiment Videos

Last Updated: Jul 10, 2026

Deep Neural Networks for Image-Based Dietary Assessment

Deep Neural Networks for Image-Based Dietary Assessment

Published on: March 13, 2021

Area of Science:

Artificial Intelligence
Machine Learning
Neural Networks

Background:

Pi-sigma networks are feedforward neural networks characterized by product units in their output layer.
Online gradient algorithms are commonly used for training feedforward neural networks.
A challenge exists in training pi-sigma networks with online gradient algorithms, leading to slow convergence due to small weight update increments, particularly in early training stages.

Discussion:

This study addresses the slow convergence issue in training pi-sigma networks.
An adaptive penalty term is introduced into the error function to mitigate the problem.
The penalty term aims to amplify small weight update increments, especially during the initial phases of training.

Key Insights:

The proposed adaptive penalty term effectively increases the magnitude of weight updates when they become excessively small.
This strategy leads to significantly faster convergence rates for pi-sigma networks compared to standard online gradient methods.
Numerical experiments validate the effectiveness of the adaptive penalty approach in accelerating network training.

Outlook:

Further research could explore variations of adaptive penalty functions for different neural network architectures.
Investigating the impact of the adaptive penalty on generalization performance is a potential future direction.
Optimizing the parameters of the adaptive penalty term could lead to even greater improvements in training efficiency.