Related Concept Videos
Distributed Loads: Problem Solving
Distributed Loads
For example, consider a bookshelf filled with books stacked vertically adjacent to each other. The weight of the books is evenly distributed over the length of the shelf. As a result, the pressure at different locations on the surface of the...
Short-distance Transport of Resources
Propagation of Uncertainty from Random Error
Associative Learning
Classical conditioning, also known...
Parallel Processing
You might also read
Related Articles
Articles linked to this work by shared authors, journal, and citation graph.
Unsupervised Neural Beamforming for Uplink MU-SIMO in 3GPP-Compliant Wireless Channels.
Meta-learning based blind image super-resolution approach to different degradations.
Blind Super-Resolution via Meta-Learning and Markov Chain Monte Carlo Simulation.
Related Experiment Video
Updated: Nov 27, 2025

Large Scale Energy Efficient Sensor Network Routing Using a Quantum Processor Unit
Published on: September 8, 2023
Straggler-Aware Distributed Learning: Communication-Computation Latency Trade-Off.
Emre Ozfatura1, Sennur Ulukus2, Deniz Gündüz1
1Information Processing and Communications Lab, Department of Electrical and Electronic Engineering, Imperial College London, London SW7 2AZ, UK.
This study introduces multi-message communication (MMC) to address straggling workers in large-scale machine learning. MMC improves efficiency by allowing workers to send multiple updates, reducing over-computation and under-utilization.
Area of Science:
- Computer Science
- Machine Learning
- Distributed Systems
Background:
- Large-scale machine learning relies on parallel gradient descent (GD).
- Straggling workers significantly limit GD's iteration speed.
- Existing methods for straggler tolerance cause over-computation or under-utilization.
Purpose of the Study:
- To overcome limitations of current straggler avoidance techniques in distributed GD.
- To introduce and analyze multi-message communication (MMC) for enhanced worker efficiency.
- To balance computation and communication latency in large-scale ML.
Main Methods:
- Proposed novel straggler avoidance techniques for coded computation and coded communication with MMC.
- Analyzed the efficiency of proposed designs for balancing computation and communication latency.
- Conducted extensive simulations, including model-based and real-world implementations on Amazon EC2.
Main Results:
- Multi-message communication (MMC) effectively addresses over-computation and under-utilization issues.
- Proposed coded computation and communication schemes with MMC demonstrate improved performance.
- Simulations confirm the benefits of MMC over existing straggler avoidance schemes.
Conclusions:
- MMC is a viable strategy to mitigate straggling worker impact in distributed GD.
- The proposed techniques offer a better balance between computation and communication efficiency.
- This work advances straggler avoidance in large-scale machine learning applications.
