Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Experiment Videos

Statistical efficiency of adaptive algorithms.

Bernard Widrow1, Max Kamenetsky

  • 1ISL, Department of Electrical Engineering, Stanford University, Rm. 273, Packard Electrical Bldg. 350 Serra Mall, Stanford, CA 94305, USA. widrow@stanford.edu

Neural Networks : the Official Journal of the International Neural Network Society
|July 10, 2003
PubMed
Summary
This summary is machine-generated.

Related Concept Videos

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Editorial: Celebrating 30 Years of IJNS and the Next Industrial Revolution.

International journal of neural systems·2021
Same author

Correction: Design and Validation of a Periodic Leg Movement Detector.

PloS one·2015
Same author

Design and validation of a periodic leg movement detector.

PloS one·2014
Same author

Special issue.

International journal of neural systems·2014
Same author

Reply to the Comments on the "No-Prop" algorithm.

Neural networks : the official journal of the International Neural Network Society·2013
Same author

Cognitive memory.

Neural networks : the official journal of the International Neural Network Society·2013
Same journal

Dynamic analysis and reliable mechanical optimization application of ring HNN effected with a memristive neuron.

Neural networks : the official journal of the International Neural Network Society·2026
Same journal

DAFF-Net: A detection and search method for small-scale low surface brightness galaxies.

Neural networks : the official journal of the International Neural Network Society·2026
Same journal

Quasi-synchronization for complex networks with hybrid pinning intermittent control.

Neural networks : the official journal of the International Neural Network Society·2026
Same journal

Physics-encoded convolutional neural operators for parametric PDEs: A convergence-guaranteed framework via pre-computed kernel fields.

Neural networks : the official journal of the International Neural Network Society·2026
Same journal

Exploiting audio-visual modalities in videos: Object detection via multi-stage bilateral coupling network.

Neural networks : the official journal of the International Neural Network Society·2026
Same journal

Reliability-aware modality completion with cross-modal distillation for federated learning with missing modalities.

Neural networks : the official journal of the International Neural Network Society·2026
See all related articles

The Least Mean Squares (LMS) algorithm offers practical implementation and optimal performance in many scenarios, making it widely applicable. While LMS/Newton is mathematically optimal, LMS provides comparable results in most real-world adaptive filtering applications.

Area of Science:

  • Signal Processing
  • Machine Learning
  • Adaptive Algorithms

Background:

  • Statistical efficiency measures learning algorithm performance based on solution quality and data usage.
  • Gradient descent adaptive algorithms are crucial for weight adaptation in various applications.
  • The Least Mean Squares (LMS) algorithm is widely used due to its simplicity and practicality.

Purpose of the Study:

  • To compare the statistical efficiency and performance of the LMS algorithm and the LMS/Newton algorithm.
  • To establish LMS/Newton as a benchmark for least squares adaptive algorithms due to its optimality.
  • To identify the practical advantages and disadvantages of both LMS and LMS/Newton algorithms.

Main Methods:

  • Defined statistical efficiency as the ratio of solution quality to training data used.

Related Experiment Videos

  • Compared the performance of LMS and LMS/Newton algorithms using ensemble learning experiences.
  • Analyzed algorithm performance under stationary and nonstationary input signals, and varying initial conditions.

    • Main Results:

    • LMS/Newton is optimal in the least squares sense, maximizing solution quality while minimizing data usage.
    • Under many conditions, including nonstationary signals, LMS and LMS/Newton exhibit equal average performance.
    • LMS/Newton generally converges faster than LMS under worst-case initial conditions.

    Conclusions:

    • LMS/Newton serves as a theoretical benchmark for least squares adaptive algorithms.
    • LMS's ease of implementation and strong performance in practical conditions contribute to its widespread use in applications like echo cancellation and channel equalization.
    • LMS is related to the backpropagation algorithm used in neural networks, highlighting its foundational importance.