Search research articles

ABOUT JoVE

Overview Leadership Blog JoVE Help Center

AUTHORS

Publishing Process Editorial Board Scope & Policies Peer Review FAQ Submit

LIBRARIANS

Testimonials Subscriptions Access Resources Library Advisory Board FAQ

RESEARCH

JoVE Journal Methods Collections JoVE Encyclopedia of Experiments Archive

EDUCATION

JoVE Core JoVE Business JoVE Science Education JoVE Lab Manual Faculty Resource Center Faculty Site

Terms & Conditions of Use

Related Concept Videos

Distributed Loads: Problem Solving

Distributed Loads: Problem Solving

Beams are structural elements commonly employed in engineering applications requiring different load-carrying capacities. The first step in analyzing a beam under a distributed load is to simplify the problem by dividing the load into smaller regions, which allows one to consider each region separately and calculate the magnitude of the equivalent resultant load acting on each portion of the beam. The magnitude of the equivalent resultant load for each region can be determined by calculating...

Distributed Loads

Distributed Loads

Distributed loads are a common type of load that engineers and scientists encounter in various practical situations. Distributed loads often refer to a type of load spread over a surface or a structure and can be modeled as continuous force per unit area.
For example, consider a bookshelf filled with books stacked vertically adjacent to each other. The weight of the books is evenly distributed over the length of the shelf. As a result, the pressure at different locations on the surface of the...

Short-distance Transport of Resources

Short-distance Transport of Resources

Short-distance transport refers to transport that occurs over a distance of just 2-3 cells, crossing the plasma membrane in the process. Small uncharged molecules, such as oxygen, carbon dioxide, and water, can diffuse across the plasma membrane on their own. In contrast, ions and larger molecules require the assistance of transport proteins due to their charge or size. Transport across membranes also occurs within individual cells, playing a variety of essential roles for the plant as a whole.

Propagation of Uncertainty from Random Error

Propagation of Uncertainty from Random Error

An experiment often consists of more than a single step. In this case, measurements at each step give rise to uncertainty. Because the measurements occur in successive steps, the uncertainty in one step necessarily contributes to that in the subsequent step. As we perform statistical analysis on these types of experiments, we must learn to account for the propagation of uncertainty from one step to the next. The propagation of uncertainty depends on the type of arithmetic operation performed on...

Associative Learning

Associative Learning

Associative learning is a fundamental concept in behavioral psychology, wherein a connection is established between two stimuli or events, leading to a learned response. This process is critical in understanding how behaviors are acquired and modified. Conditioning, the mechanism through which associations are formed, can be divided into two main types: classical conditioning and operant conditioning, each elucidating different aspects of associative learning.
Classical conditioning, also known...

Parallel Processing

Parallel Processing

The brain processes sensory information rapidly due to parallel processing, which involves sending data across multiple neural pathways at the same time. This method allows the brain to manage various sensory qualities, such as shapes, colors, movements, and locations, all concurrently. For instance, when observing a forest landscape, the brain simultaneously processes the movement of leaves, the shapes of trees, the depth between them, and the various shades of green. This enables a quick and...

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by

Same author

Unsupervised Neural Beamforming for Uplink MU-SIMO in 3GPP-Compliant Wireless Channels.

Sensors (Basel, Switzerland)·2026

Same author

Meta-learning based blind image super-resolution approach to different degradations.

Neural networks : the official journal of the International Neural Network Society·2024

Same author

Blind Super-Resolution via Meta-Learning and Markov Chain Monte Carlo Simulation.

IEEE transactions on pattern analysis and machine intelligence·2024

Same author

Deceptive Information Retrieval.

Entropy (Basel, Switzerland)·2024

Same author

PRIMIS: Privacy-preserving medical image sharing via deep sparsifying transform learning with obfuscation.

Journal of biomedical informatics·2024

Same author

Multi-institutional PET/CT image segmentation using federated deep transformer learning.

Computer methods and programs in biomedicine·2023

Search research articles

Related Experiment Video

Updated: Nov 27, 2025

Large Scale Energy Efficient Sensor Network Routing Using a Quantum Processor Unit

Large Scale Energy Efficient Sensor Network Routing Using a Quantum Processor Unit

Published on: September 8, 2023

Straggler-Aware Distributed Learning: Communication-Computation Latency Trade-Off.

Emre Ozfatura¹, Sennur Ulukus², Deniz Gündüz¹

¹Information Processing and Communications Lab, Department of Electrical and Electronic Engineering, Imperial College London, London SW7 2AZ, UK.

Entropy (Basel, Switzerland)

|December 8, 2020

Summary

This study introduces multi-message communication (MMC) to address straggling workers in large-scale machine learning. MMC improves efficiency by allowing workers to send multiple updates, reducing over-computation and under-utilization.

Keywords:

coded computation distributed computation gradient coding gradient descent machine learning parallel computing polynomial codes

More Related Videos

Quasi-light Storage for Optical Data Packets

Quasi-light Storage for Optical Data Packets

Published on: February 6, 2014

Related Experiment Videos

Last Updated: Nov 27, 2025

Large Scale Energy Efficient Sensor Network Routing Using a Quantum Processor Unit

Large Scale Energy Efficient Sensor Network Routing Using a Quantum Processor Unit

Published on: September 8, 2023

Quasi-light Storage for Optical Data Packets

Quasi-light Storage for Optical Data Packets

Published on: February 6, 2014

Area of Science:

Computer Science
Machine Learning
Distributed Systems

Background:

Large-scale machine learning relies on parallel gradient descent (GD).
Straggling workers significantly limit GD's iteration speed.
Existing methods for straggler tolerance cause over-computation or under-utilization.

Purpose of the Study:

To overcome limitations of current straggler avoidance techniques in distributed GD.
To introduce and analyze multi-message communication (MMC) for enhanced worker efficiency.
To balance computation and communication latency in large-scale ML.

Main Methods:

Proposed novel straggler avoidance techniques for coded computation and coded communication with MMC.
Analyzed the efficiency of proposed designs for balancing computation and communication latency.
Conducted extensive simulations, including model-based and real-world implementations on Amazon EC2.

Main Results:

Multi-message communication (MMC) effectively addresses over-computation and under-utilization issues.
Proposed coded computation and communication schemes with MMC demonstrate improved performance.
Simulations confirm the benefits of MMC over existing straggler avoidance schemes.

Conclusions:

MMC is a viable strategy to mitigate straggling worker impact in distributed GD.
The proposed techniques offer a better balance between computation and communication efficiency.
This work advances straggler avoidance in large-scale machine learning applications.