Fast curvature matrix-vector products for second-order gradient descent

Nicol N Schraudolph1

  • 1IDSIA, Galleria 2, 6928 Manno, Switzerland. nic@inf.ethz.ch

Neural Computation
|June 25, 2002
PubMed

Related Concept Videos

Second Derivatives and Laplace Operator01:22

Second Derivatives and Laplace Operator

The first order operators using the del operator include the gradient, divergence and curl. Certain combinations of first order operators on a scalar or vector function yield second order expressions. Second-order expressions play a very important role in mathematics and physics. Some second order expressions include the divergence and curl of a gradient function, the divergence and curl of a curl function, and the gradient of a divergence function.
Consider a scalar function. The curl of its...
Gradient Fields01:27

Gradient Fields

A gradient field is a vector field derived from a scalar field. A scalar field assigns a single numerical value to every point in space, such as temperature, pressure, or electric potential. The gradient field describes how that value changes from point to point. It gives both the direction of the fastest increase and the rate of change in that direction.For a scalar field f(x, y), the gradient is written as\begin{equation*}\nabla f=\left\langle \jfrac{\partial f}{\partial x},\jfrac{\partial...
Gradient Vectors and Their Applications01:19

Gradient Vectors and Their Applications

Every point on a topographical map corresponds to a particular elevation, so the landscape can be modeled as a surface whose height depends on horizontal position. From any given location, a hiker may face infinitely many directions, but only one direction produces the fastest possible increase in elevation. This unique route is called the direction of steepest ascent, and in multivariable calculus, it is represented by the gradient vector of the elevation function.The gradient vector points...
Vector Transformation in Rotating Coordinate Systems01:16

Vector Transformation in Rotating Coordinate Systems

Consider a vector rotating about an axis with an angular velocity, such that its tip sweeps a circular path.
Gradient and Del Operator01:14

Gradient and Del Operator

In mathematics and physics, the gradient and del operator are fundamental concepts used to describe the behavior of functions and fields in space. The gradient is a mathematical operator that gives both the magnitude and direction of the maximum spatial rate of change. Consider a person standing on a mountain. The slope of the mountain at any given point is not defined unless it is quantified in a particular direction. For this reason, a "directional derivative" is defined, which is a vector...
Derivatives of Vector Functions01:17

Derivatives of Vector Functions

A vector-valued function describes position as a function of time. For example, in Cartesian coordinates, the position of a car moving along a curved road can be written as\begin{equation*}\textbf{r}(t)=\langle x(t),y(t),z(t)\rangle\end{equation*}Secant Vector and Average Velocity:This secant vector captures the overall change in position during the interval and provides a crude estimate of the direction of motion.At time t, the car is at point P, with position r(t). After a short interval h,...