Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

Mechanistic Models: Compartment Models in Algorithms for Numerical Problem Solving01:29

Mechanistic Models: Compartment Models in Algorithms for Numerical Problem Solving

Mechanistic models play a crucial role in algorithms for numerical problem-solving, particularly in nonlinear mixed effects modeling (NMEM). These models aim to minimize specific objective functions by evaluating various parameter estimates, leading to the development of systematic algorithms. In some cases, linearization techniques approximate the model using linear equations.
In individual population analyses, different algorithms are employed, such as Cauchy's method, which uses a...
Lagrange Multipliers: Problem Solving01:30

Lagrange Multipliers: Problem Solving

A silo with a cylindrical base, flat bottom, and hemispherical roof is a common design in agricultural and industrial storage due to its structural efficiency and ease of construction. Optimizing its dimensions to maximize storage capacity for a given amount of material—i.e., a fixed surface area—is a classic problem in applied calculus and engineering design. The key parameters are the radius r of the base and the height h of the cylindrical section.The total volume of the silo is obtained by...
Linear Approximations01:23

Linear Approximations

For a differentiable function of two variables, linear approximation estimates values near a known point by replacing the curved surface with its tangent plane. Consider the function\begin{equation*}f(x,y)=x^2+3y^2\end{equation*}near the point (2, 1). The exact value at this point is f(2, 1) = 22 + 3(1)2 = 4 + 3 = 7.The linear approximation of f(x, y)) near (a, b) is\begin{equation*}L(x,y)=f(a,b)+f_x(a,b)(x-a)+f_y(a,b)(y-b)\end{equation*}First, compute the partial derivatives: fx(x, y) = 2x and...
Linearization and Approximation01:26

Linearization and Approximation

Linearization is a mathematical technique used to approximate complex, nonlinear functions with simpler linear models in the vicinity of a chosen reference point. The method is based on the idea that, although a function may be difficult to evaluate exactly, its behavior near a specific input value can often be closely approximated by the tangent line at that point. This approach is particularly useful when small deviations from a known value are involved.Consider the square root function, for...
Improving Translational Accuracy02:07

Improving Translational Accuracy

Base complementarity between the three base pairs of mRNA codon and the tRNA anticodon is not a failsafe mechanism. Inaccuracies can range from a single mismatch to no correct base pairing at all. The free energy difference between the correct and nearly correct base pairs can be as small as 3 kcal/ mol. With complementarity being the only proofreading step, the estimated error frequency would be one wrong amino acid in every 100 amino acids incorporated. However, error frequencies observed in...
Improving Translational Accuracy02:07

Improving Translational Accuracy

Base complementarity between the three base pairs of mRNA codon and the tRNA anticodon is not a failsafe mechanism. Inaccuracies can range from a single mismatch to no correct base pairing at all. The free energy difference between the correct and nearly correct base pairs can be as small as 3 kcal/ mol. With complementarity being the only proofreading step, the estimated error frequency would be one wrong amino acid in every 100 amino acids incorporated. However, error frequencies observed in...

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Neural Network Training With Levenberg-Marquardt and Adaptable Weight Compression.

IEEE transactions on neural networks and learning systems·2018
Same author

Autophagy in long propriospinal neurons is activated after spinal cord injury in adult rats.

Neuroscience letters·2016
Same author

Therapeutic effect of apatinib on overall survival is mediated by prolonged progression-free survival in advanced gastric cancer patients.

Oncotarget·2016
Same author

Prevalence of hemorrhagic fever with renal syndrome in Qingdao City, China, 2010-2014.

Scientific reports·2016
Same author

Neutralizing Antibodies to Severe Fever with Thrombocytopenia Syndrome Virus 4 Years after Hospitalization, China.

Emerging infectious diseases·2016
Same author

Elevated Platelet to Lymphocyte Ratio Is Associated with Poor Survival Outcomes in Patients with Colorectal Cancer.

PloS one·2016

Related Experiment Videos

Improved computation for Levenberg-Marquardt training.

Bogdan M Wilamowski1, Hao Yu

  • 1Department of Electrical and Computer Engineering, Auburn University, Auburn, AL 36849-5201, USA. wilam@ieee.org

IEEE Transactions on Neural Networks
|April 23, 2010
PubMed
Summary
This summary is machine-generated.

This study optimizes neural network training using the Levenberg-Marquardt (LM) algorithm by directly computing the quasi-Hessian matrix. This significantly improves memory and time efficiency, especially for large datasets.

Related Experiment Videos

Area of Science:

  • Computational neuroscience
  • Machine learning algorithms

Background:

  • The Levenberg-Marquardt (LM) algorithm is a standard iterative technique for solving nonlinear least squares problems, commonly used in neural network training.
  • Traditional LM implementations require significant memory for Jacobian matrix storage and computation, limiting scalability for large-scale problems.

Purpose of the Study:

  • To enhance the computational efficiency of the Levenberg-Marquardt (LM) algorithm for neural network training.
  • To address the memory limitations associated with conventional LM algorithm implementations.

Main Methods:

  • Developed a novel computational approach for the LM algorithm that bypasses explicit Jacobian matrix multiplication and storage.
  • Introduced direct computation of the quasi-Hessian matrix and gradient vector.
  • Exploited the symmetry of the quasi-Hessian matrix to reduce computational load by calculating only upper/lower triangular elements.

Main Results:

  • Achieved significant improvements in training speed due to reduced memory footprint and fewer operations.
  • Successfully resolved the memory limitation problem inherent in standard LM training.
  • Demonstrated particularly enhanced memory and time efficiencies when training large-sized patterns.

Conclusions:

  • The proposed direct computation method offers a more efficient alternative for LM-based neural network training.
  • This optimization is crucial for enabling the training of larger and more complex neural network models.
  • The findings pave the way for broader applications of the LM algorithm in resource-constrained environments.