Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

Mechanistic Models: Compartment Models in Algorithms for Numerical Problem Solving01:29

Mechanistic Models: Compartment Models in Algorithms for Numerical Problem Solving

Mechanistic models play a crucial role in algorithms for numerical problem-solving, particularly in nonlinear mixed effects modeling (NMEM). These models aim to minimize specific objective functions by evaluating various parameter estimates, leading to the development of systematic algorithms. In some cases, linearization techniques approximate the model using linear equations.
In individual population analyses, different algorithms are employed, such as Cauchy's method, which uses a...
Linear Approximations01:23

Linear Approximations

For a differentiable function of two variables, linear approximation estimates values near a known point by replacing the curved surface with its tangent plane. Consider the function\begin{equation*}f(x,y)=x^2+3y^2\end{equation*}near the point (2, 1). The exact value at this point is f(2, 1) = 22 + 3(1)2 = 4 + 3 = 7.The linear approximation of f(x, y)) near (a, b) is\begin{equation*}L(x,y)=f(a,b)+f_x(a,b)(x-a)+f_y(a,b)(y-b)\end{equation*}First, compute the partial derivatives: fx(x, y) = 2x and...
Gradient Vectors and Their Applications01:19

Gradient Vectors and Their Applications

Every point on a topographical map corresponds to a particular elevation, so the landscape can be modeled as a surface whose height depends on horizontal position. From any given location, a hiker may face infinitely many directions, but only one direction produces the fastest possible increase in elevation. This unique route is called the direction of steepest ascent, and in multivariable calculus, it is represented by the gradient vector of the elevation function.The gradient vector points...
Application of Linearization and Approximation01:29

Application of Linearization and Approximation

A drone flying through complex terrain often relies on more than one sensing method to estimate small changes in altitude. Along with direct measurements, air pressure provides a useful indirect indicator of vertical movement. Atmospheric pressure decreases as altitude increases, and this relationship is commonly described using an exponential model. Although accurate, converting pressure measurements into altitude values requires calculations that are too complex to perform repeatedly during...
Lagrange Multipliers: Two Constraints01:28

Lagrange Multipliers: Two Constraints

The method of Lagrange multipliers with two constraints is used to optimize a function subject to two independent constraints. In many applications, the objective function represents a quantity to be maximized or minimized, such as cost, area, distance, or energy. The two constraints represent requirements that the solution must satisfy, such as fixed volume, limited resources, or prescribed dimensions.For a function of three variables, each constraint forms a surface in three-dimensional space.
Linearization and Approximation01:26

Linearization and Approximation

Linearization is a mathematical technique used to approximate complex, nonlinear functions with simpler linear models in the vicinity of a chosen reference point. The method is based on the idea that, although a function may be difficult to evaluate exactly, its behavior near a specific input value can often be closely approximated by the tangent line at that point. This approach is particularly useful when small deviations from a known value are involved.Consider the square root function, for...

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Predicting scientific breakthroughs through capacity-based dynamics in citation networks.

Physical review. E·2026
Same author

Effect of substituent volume on the adsorption and separation performance of thermo-responsive ion-imprinted polymers.

Analytica chimica acta·2026
Same author

Efficacy and safety of six immunoadsorption treatments for severe lupus nephritis: a Bayesian network meta-analysis and systematic review.

Frontiers in immunology·2026
Same author

Simultaneous inhibition of bacterial virulence and anti-phage defense systems by synergistic bacteriophage counter-defense proteins.

The EMBO journal·2026
Same author

Pan-immune-inflammation value as an independent prognostic marker in patients with brain metastases.

Frontiers in oncology·2026
Same author

Efficacy comparison of seven non-invasive brain stimulation techniques for upper limb motor dysfunction after stroke: a Bayesian network meta-analysis and systematic review.

Frontiers in neurology·2026
Same journal

Application of ephrin-B2 loaded glycol chitosan-silk fibroin hydrogel in the treatment of diabetic refractory wounds.

Scientific reports·2026
Same journal

International expert Delphi consensus on thromboprophylaxis in metabolic and bariatric surgery.

Scientific reports·2026
Same journal

Assessing the cross-region knowledge transfer capability of selected deep learning building vectorization methods in the context of available training datasets.

Scientific reports·2026
Same journal

Feasibility and preliminary effects of outdoor versus indoor cognitive-motor therapy in women with Alzheimer's disease: A randomized single-blind pilot study.

Scientific reports·2026
Same journal

Hallmarks of social action in the vocal turn-taking of wild common marmosets (Callithrix jacchus).

Scientific reports·2026
Same journal

Role and mechanism of AOPPs-induced NOX4-mediated ferroptosis in intervertebral disc degeneration.

Scientific reports·2026
See all related articles

Related Experiment Video

Updated: Jun 24, 2026

Deep Neural Networks for Image-Based Dietary Assessment
13:19

Deep Neural Networks for Image-Based Dietary Assessment

Published on: March 13, 2021

Stochastic three-term conjugate gradient: a third-order curvature approximation correction algorithmic framework for

Jiazhen Liu1, Gonglin Yuan2, Zhongyu Mo3

  • 1Postdoctoral Research Workstation, Guangxi Rural Commercial United Bank, No. 148 Minzu Ave., Nanning, 530022, P. R. China.

Scientific Reports
|June 22, 2026
PubMed
Summary
This summary is machine-generated.

This study introduces a novel stochastic three-term conjugate gradient algorithm (TASCG) that uses third-order curvature information. TASCG enhances convergence in nonconvex optimization by better capturing landscape geometry, matching optimal stochastic gradient method complexity.

Keywords:
Nonconvex optimizationStochastic optimizationThird-order curvature approximationThree-term conjugate gradient methodVariance reduction

Related Experiment Videos

Last Updated: Jun 24, 2026

Deep Neural Networks for Image-Based Dietary Assessment
13:19

Deep Neural Networks for Image-Based Dietary Assessment

Published on: March 13, 2021

Area of Science:

  • Optimization Theory
  • Machine Learning Algorithms
  • Numerical Analysis

Background:

  • Classical conjugate gradient methods utilize only first-order information, limiting their effectiveness in capturing curvature for nonconvex stochastic optimization.
  • Nonconvex loss landscapes present challenges for optimization algorithms due to complex geometric properties.
  • Existing stochastic methods often struggle with convergence speed and accuracy in complex optimization scenarios.

Purpose of the Study:

  • To propose a novel stochastic three-term conjugate gradient algorithm (TASCG) that incorporates third-order curvature approximation.
  • To enhance the ability of optimization algorithms to capture local geometry in nonconvex loss landscapes.
  • To establish theoretical convergence guarantees and analyze the oracle complexity of the proposed algorithm.

Main Methods:

  • Development of the stochastic three-term conjugate gradient algorithm (TASCG) integrating third-order tensor information into the search direction.
  • Theoretical analysis establishing global convergence under standard assumptions (gradient Lipschitz continuity, bounded variance).
  • Derivation of a stochastic first-order oracle complexity bound matching optimal classical methods.
  • Incorporation of variance reduction techniques to create the TASCG-VR variant for improved gradient estimation.

Main Results:

  • The TASCG algorithm successfully integrates third-order curvature information, improving the capture of local geometry in nonconvex landscapes.
  • Global convergence of TASCG is rigorously established, with a derived oracle complexity bound matching optimal stochastic gradient methods.
  • Numerical experiments show TASCG and TASCG-VR significantly outperform standard Stochastic Gradient Descent (SGD) and Stochastic Variance Reduced Gradient (SVRG) in convergence speed and accuracy.
  • The proposed algorithms demonstrate increased robustness to step-size selection compared to existing methods.

Conclusions:

  • The proposed TASCG algorithm offers a novel framework for integrating higher-order geometric information into stochastic conjugate gradient methods.
  • TASCG provides a theoretically sound and practically effective approach for nonconvex stochastic optimization.
  • The findings suggest a promising direction for developing more efficient and robust optimization algorithms in machine learning and related fields.