Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

Types of Errors: Detection and Minimization01:12

Types of Errors: Detection and Minimization

1.2K
Error is the deviation of the obtained result from the true, expected value or the estimated central value. Errors are expressed in absolute or relative terms.
Absolute error in a measurement is the numerical difference from the true or central value. Relative error is the ratio between absolute error and the true or central value, expressed as a percentage.
Errors can be classified by source, magnitude, and sign. There are three types of errors: systematic, random, and gross.
Systematic or...
1.2K
Propagation of Uncertainty from Random Error00:59

Propagation of Uncertainty from Random Error

500
An experiment often consists of more than a single step. In this case, measurements at each step give rise to uncertainty. Because the measurements occur in successive steps, the uncertainty in one step necessarily contributes to that in the subsequent step. As we perform statistical analysis on these types of experiments, we must learn to account for the propagation of uncertainty from one step to the next. The propagation of uncertainty depends on the type of arithmetic operation performed on...
500
Propagation of Uncertainty from Systematic Error01:10

Propagation of Uncertainty from Systematic Error

345
The atomic mass of an element varies due to the relative ratio of its isotopes. A sample's relative proportion of oxygen isotopes influences its average atomic mass. For instance, if we were to measure the atomic mass of oxygen from a sample, the mass would be a weighted average of the isotopic masses of oxygen in that sample. Since a single sample is not likely to perfectly reflect the true atomic mass of oxygen for all the molecules of oxygen on Earth, the mass we obtain from this...
345
Learning Disabilities01:25

Learning Disabilities

64
Learning disabilities are cognitive disorders caused by neurological impairments that affect cognitive functions like language and reading, without indicating overall intellectual or developmental challenges. These disabilities differ from global intellectual or developmental disabilities as they are limited to distinct cognitive functions. Common learning disabilities include dysgraphia, dyslexia, and dyscalculia, each of which impacts unique aspects of learning.
Dyslexia
Dyslexia is a...
64
Distance Corrections01:15

Distance Corrections

18
To achieve precise distance measurements, especially in surveying and construction, certain corrections must be applied to account for potential sources of error like the standardization errors, temperature variations, and slope adjustments.Standardization error emerges when measurement equipment undergoes changes, such as wear, repairs, or weather impacts. To address this, surveyors compare the equipment’s readings to a standard. This process identifies any deviation that might lead to...
18
Associative Learning01:27

Associative Learning

236
Associative learning is a fundamental concept in behavioral psychology, wherein a connection is established between two stimuli or events, leading to a learned response. This process is critical in understanding how behaviors are acquired and modified. Conditioning, the mechanism through which associations are formed, can be divided into two main types: classical conditioning and operant conditioning, each elucidating different aspects of associative learning.
Classical conditioning, also known...
236

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Holistic Invariant Retracing for Distortion-Resilient Multi-Modal Learning in Spatial Transcriptomics.

IEEE transactions on image processing : a publication of the IEEE Signal Processing Society·2026
Same author

Demonstration of efficient predictive surrogates for large-scale quantum processors.

Nature communications·2026
Same author

A DeepSeek-powered AI system for automated chest radiograph interpretation in clinical practice.

Nature communications·2026
Same author

NoisePO: Efficient Semantic Noise Generation and Ranking for Diffusion-Based Text-to-Image Synthesis.

IEEE transactions on image processing : a publication of the IEEE Signal Processing Society·2026
Same author

Stability and Generalization for Distributed SGDA.

IEEE transactions on pattern analysis and machine intelligence·2026
Same author

SPAgent: Adaptive Task Decomposition and Model Selection for General Video Generation and Editing.

IEEE transactions on image processing : a publication of the IEEE Signal Processing Society·2026
Same journal

Dynamic analysis and reliable mechanical optimization application of ring HNN effected with a memristive neuron.

Neural networks : the official journal of the International Neural Network Society·2026
Same journal

DAFF-Net: A detection and search method for small-scale low surface brightness galaxies.

Neural networks : the official journal of the International Neural Network Society·2026
Same journal

Quasi-synchronization for complex networks with hybrid pinning intermittent control.

Neural networks : the official journal of the International Neural Network Society·2026
Same journal

Physics-encoded convolutional neural operators for parametric PDEs: A convergence-guaranteed framework via pre-computed kernel fields.

Neural networks : the official journal of the International Neural Network Society·2026
Same journal

Exploiting audio-visual modalities in videos: Object detection via multi-stage bilateral coupling network.

Neural networks : the official journal of the International Neural Network Society·2026
Same journal

Reliability-aware modality completion with cross-modal distillation for federated learning with missing modalities.

Neural networks : the official journal of the International Neural Network Society·2026
See all related articles

Related Experiment Video

Updated: May 9, 2025

Operation of the Collaborative Composite Manufacturing CCM System
10:09

Operation of the Collaborative Composite Manufacturing CCM System

Published on: October 1, 2019

6.5K

Communication-efficient distributed learning with Local Immediate Error Compensation.

Yifei Cheng1, Li Shen2, Linli Xu3

  • 1Guangming Laboratory, China; School of Cyber Science and Technology, Shenzhen Campus of Sun Yat-sen University, China.

Neural Networks : the Official Journal of the International Neural Network Society
|April 30, 2025
PubMed
Summary
This summary is machine-generated.

Local Immediate Error Compensated SGD (LIEC-SGD) reduces communication costs in distributed learning. This novel algorithm achieves faster convergence and lower overhead than existing methods, improving deep learning model training efficiency.

Keywords:
CommunicationDistributed learningGradient compressionOptimization

More Related Videos

Large Scale Energy Efficient Sensor Network Routing Using a Quantum Processor Unit
05:30

Large Scale Energy Efficient Sensor Network Routing Using a Quantum Processor Unit

Published on: September 8, 2023

452
Measuring Statistical Learning Across Modalities and Domains in School-Aged Children Via an Online Platform and Neuroimaging Techniques
08:05

Measuring Statistical Learning Across Modalities and Domains in School-Aged Children Via an Online Platform and Neuroimaging Techniques

Published on: June 30, 2020

7.4K

Related Experiment Videos

Last Updated: May 9, 2025

Operation of the Collaborative Composite Manufacturing CCM System
10:09

Operation of the Collaborative Composite Manufacturing CCM System

Published on: October 1, 2019

6.5K
Large Scale Energy Efficient Sensor Network Routing Using a Quantum Processor Unit
05:30

Large Scale Energy Efficient Sensor Network Routing Using a Quantum Processor Unit

Published on: September 8, 2023

452
Measuring Statistical Learning Across Modalities and Domains in School-Aged Children Via an Online Platform and Neuroimaging Techniques
08:05

Measuring Statistical Learning Across Modalities and Domains in School-Aged Children Via an Online Platform and Neuroimaging Techniques

Published on: June 30, 2020

7.4K

Area of Science:

  • Machine Learning
  • Distributed Systems
  • Optimization Algorithms

Background:

  • Distributed learning faces significant communication overhead.
  • Existing gradient compression methods have limitations in either communication cost or convergence rate.

Purpose of the Study:

  • To propose a novel optimization algorithm, Local Immediate Error Compensated SGD (LIEC-SGD), to address the bottlenecks in distributed learning.
  • To reduce communication costs while maintaining or improving convergence rates.

Main Methods:

  • Implemented bidirectional compression to decrease communication load.
  • Introduced immediate local error compensation to the model update.
  • Maintained only the global error variable on the server to enhance efficiency.

Main Results:

  • LIEC-SGD theoretically outperforms previous methods in convergence rate and communication cost.
  • Experimental results show improved test accuracies on CIFAR-10 and CIFAR-100 datasets.
  • Achieved significant speedups (1.428× and 1.721×) over parallel SGD.

Conclusions:

  • LIEC-SGD effectively inherits the dual advantages of unidirectional and bidirectional compression.
  • The proposed algorithm demonstrates superior performance in both accuracy and training time for deep neural networks.