Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Experiment Videos

A pruning method for the recursive least squared algorithm.

C S Leung1, K W Wong, P F Sum

  • 1Department of Electronic Engineering, City University of Hong Kong, Kowloon Tong.

Neural Networks : the Official Journal of the International Neural Network Society
|April 24, 2001
PubMed
Summary
This summary is machine-generated.

Related Concept Videos

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

MRI evaluation of congenital vaginal malformations-a pictorial review.

Clinical radiology·2026
Same author

The incidence of arthrogenic muscle inhibition in acute knee injuries: a prospective cohort study in a tertiary acute knee unit.

Asia-Pacific journal of sports medicine, arthroscopy, rehabilitation and technology·2025
Same author

Direct Experimental Proof of the Principal Role of Reduced High-Mode Hydrodynamic Mix in Recent Ignition Success on NIF.

Physical review letters·2025
Same author

Physiological and molecular impairment of PV circuit homeostasis in mouse models of autism.

bioRxiv : the preprint server for biology·2025
Same author

Time Pattern of Presentation of Victims of High-Speed Passenger Ferry Mass Casualty Incidents to the Emergency Department.

Disaster medicine and public health preparedness·2024
Same author

Achievement of Target Gain Larger than Unity in an Inertial Fusion Experiment.

Physical review letters·2024
Same journal

Q-learning based asynchronous Boolean control networks stabilization with data loss.

Neural networks : the official journal of the International Neural Network Society·2026
Same journal

New results on prescribed-time synchronization of complex networks via intermittent control.

Neural networks : the official journal of the International Neural Network Society·2026
Same journal

Variance-constrained multi-view ensemble broad network for imbalanced data.

Neural networks : the official journal of the International Neural Network Society·2026
Same journal

Dynamic analysis and reliable mechanical optimization application of ring HNN effected with a memristive neuron.

Neural networks : the official journal of the International Neural Network Society·2026
Same journal

DAFF-Net: A detection and search method for small-scale low surface brightness galaxies.

Neural networks : the official journal of the International Neural Network Society·2026
Same journal

Quasi-synchronization for complex networks with hybrid pinning intermittent control.

Neural networks : the official journal of the International Neural Network Society·2026
See all related articles

The recursive least squared (RLS) algorithm implicitly performs weight decay, controlled by the initial error covariance matrix. This method enhances neural network training and pruning by utilizing the approximate Hessian matrix for weight removal.

Area of Science:

  • Artificial Intelligence
  • Machine Learning
  • Neural Networks

Background:

  • The recursive least squared (RLS) algorithm is a known online training method for neural networks.
  • The interplay between RLS, weight decay, and pruning requires further investigation.

Purpose of the Study:

  • To explore how initial error covariance matrix values in RLS impact neural network generalization.
  • To determine the benefits of using the final error covariance matrix for neural network pruning.

Main Methods:

  • Investigated the implicit weight decay mechanism within the RLS algorithm.
  • Established the relationship between the inverse error covariance matrix and the Hessian matrix.
  • Proposed a training and pruning strategy using RLS and approximate Hessian-based weight removal.

Related Experiment Videos

Main Results:

  • The RLS algorithm inherently acts as a weight decay method, modulated by the initial error covariance matrix.
  • The inverse of the error covariance matrix approximates the Hessian matrix during training.
  • The proposed method demonstrated effectiveness in neural network training and pruning.

Conclusions:

  • Selecting an appropriate initial error covariance matrix value improves neural network generalization.
  • Utilizing the final error covariance matrix aids in effective neural network pruning.
  • The RLS algorithm combined with Hessian-based pruning offers an efficient approach for neural network optimization.