Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

Vector Algebra: Method of Components01:08

Vector Algebra: Method of Components

17.2K
It is cumbersome to find the magnitudes of vectors using the parallelogram rule or using the graphical method to perform mathematical operations like addition, subtraction, and multiplication. There are two ways to circumvent this algebraic complexity. One way is to draw the vectors to scale, as in navigation, and read approximate vector lengths and angles (directions) from the graphs. The other way is to use the method of components.
In many applications, the magnitudes and directions of...
17.2K
Vector Algebra: Graphical Method01:10

Vector Algebra: Graphical Method

15.4K
Vectors can be multiplied by scalars, added to other vectors, or subtracted from other vectors. The vector sum of two (or more) vectors is called the resultant vector or, for short, the resultant.
We use the laws of geometry to construct resultant vectors, followed by trigonometry to find vector magnitudes and directions. For a geometric construction of the sum of two vectors in a plane, we follow the parallelogram rule. Suppose two vectors are at arbitrary positions. Translate either one of...
15.4K
Divergence and Stokes' Theorems01:06

Divergence and Stokes' Theorems

2.5K
The divergence and Stokes' theorems are a variation of Green's theorem in a higher dimension. They are also a generalization of the fundamental theorem of calculus. The divergence theorem and Stokes' theorem are in a way similar to each other; The divergence theorem relates to the dot product of a vector, while Stokes' theorem relates to the curl of a vector. Many applications in physics and engineering make use of the divergence and Stokes' theorems, enabling us to write...
2.5K
One-Compartment Open Model: Wagner-Nelson and Loo Riegelman Method for ka Estimation01:24

One-Compartment Open Model: Wagner-Nelson and Loo Riegelman Method for ka Estimation

798
This lesson introduces two critical methods in pharmacokinetics, the Wagner-Nelson and Loo-Riegelman methods, used for estimating the absorption rate constant (ka) for drugs administered via non-intravenous routes. The Wagner-Nelson method relates ka to the plasma concentration derived from the slope of a semilog percent unabsorbed time plot. However, it is limited to drugs with one-compartment kinetics and can be impacted by factors like gastrointestinal motility or enzymatic degradation.
On...
798
Gauss's Law: Problem-Solving01:10

Gauss's Law: Problem-Solving

2.2K
Gauss's law helps determine electric fields even though the law is not directly about electric fields but electric flux. In situations with certain symmetries (spherical, cylindrical, or planar) in the charge distribution, the electric field can be deduced based on the knowledge of the electric flux. In these systems, we can find a Gaussian surface S over which the electric field has a constant magnitude. Furthermore, suppose the electric field is parallel (or antiparallel) to the area...
2.2K
Second Derivatives and Laplace Operator01:22

Second Derivatives and Laplace Operator

2.0K
The first order operators using the del operator include the gradient, divergence and curl. Certain combinations of first order operators on a scalar or vector function yield second order expressions. Second-order expressions play a very important role in mathematics and physics. Some second order expressions include the divergence and curl of a gradient function, the divergence and curl of a curl function, and the gradient of a divergence function.
Consider a scalar function. The curl of its...
2.0K

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Evaluating automatic speech recognition systems as quantitative models of cross-lingual phonetic category perception.

The Journal of the Acoustical Society of America·2018
Same author

SegAnnDB: interactive Web-based genomic segmentation.

Bioinformatics (Oxford, England)·2014
Same author

Learning smoothing models of copy number profiles using breakpoint annotations.

BMC bioinformatics·2013
Same author

Task-driven dictionary learning.

IEEE transactions on pattern analysis and machine intelligence·2011
Same author

A Tensor-Based Algorithm for High-Order Graph Matching.

IEEE transactions on pattern analysis and machine intelligence·2011
Same author

A path following algorithm for the graph matching problem.

IEEE transactions on pattern analysis and machine intelligence·2009

Related Experiment Video

Updated: Oct 15, 2025

Visualization Method for Proprioceptive Drift on a 2D Plane Using Support Vector Machine
07:05

Visualization Method for Proprioceptive Drift on a 2D Plane Using Support Vector Machine

Published on: October 27, 2016

9.3K

Stochastic quasi-gradient methods: variance reduction via Jacobian sketching.

Robert M Gower1, Peter Richtárik2,3,4, Francis Bach5

  • 1LTCI, Telécom Paris, Institut Polytechnique de Paris, Palaiseau, France.

Mathematical Programming
|November 1, 2021
PubMed
Summary

We introduce JacSketch, a novel variance-reduced stochastic gradient descent method for large-scale optimization. It efficiently estimates the Jacobian matrix using randomized linear algebra, achieving linear convergence for smooth and strongly convex functions.

Keywords:
CovariatesSketchingStochastic gradient descentVariance reduction

More Related Videos

Deep Neural Networks for Image-Based Dietary Assessment
13:19

Deep Neural Networks for Image-Based Dietary Assessment

Published on: March 13, 2021

9.4K

Related Experiment Videos

Last Updated: Oct 15, 2025

Visualization Method for Proprioceptive Drift on a 2D Plane Using Support Vector Machine
07:05

Visualization Method for Proprioceptive Drift on a 2D Plane Using Support Vector Machine

Published on: October 27, 2016

9.3K
Deep Neural Networks for Image-Based Dietary Assessment
13:19

Deep Neural Networks for Image-Based Dietary Assessment

Published on: March 13, 2021

9.4K

Area of Science:

  • Optimization
  • Machine Learning
  • Numerical Linear Algebra

Background:

  • Minimizing the average of a large number of smooth functions is a common problem in machine learning.
  • Existing methods like stochastic gradient descent can suffer from slow convergence or high variance.

Purpose of the Study:

  • To develop a new family of variance-reduced stochastic gradient descent (SGD) methods for large-scale optimization.
  • To improve the efficiency and convergence rates of optimization algorithms.

Main Methods:

  • JacSketch method: maintains a stochastic estimate of the Jacobian matrix using randomized linear algebra and sketching.
  • Efficiently updates the Jacobian estimate via sketching and projection.
  • Computes a variance-reduced unbiased gradient estimator.
  • Analogous to quasi-Newton methods, termed a stochastic quasi-gradient method.

Main Results:

  • Proved linear convergence for smooth and strongly convex functions with a general convergence theorem for sketches.
  • Developed a refined convergence theorem using a stochastic Lyapunov function for specific sketches.
  • Achieved sharper complexity results for importance sampling variants.
  • Specializing JacSketch recovers stochastic average gradient (SAGA) and its variants.
  • Established the current best-known rate for SAGA with importance sampling, resolving a conjecture.
  • Obtained superior rates for minibatch SAGA, showing decreased complexity with increased batch size.
  • Introduced the first minibatch SAGA method with importance sampling.

Conclusions:

  • JacSketch offers a powerful and versatile framework for large-scale optimization.
  • The method provides theoretical guarantees and practical improvements over existing techniques.
  • It unifies and extends several existing optimization algorithms, including SAGA.