DPro-SM - A distributed framework for proactive straggler mitigation using LSTM.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Computer Science

Background:

• Distributed Deep Learning (DDL) is crucial for big data and high-performance computing.
• Increasing data volume and network complexity exacerbate DDL challenges, leading to high training times and reduced accuracy.
• Straggler nodes create communication bottlenecks, significantly delaying DDL training processes.

Purpose of the Study:

• To address the communication overhead and straggler node issues in DDL.
• To propose a novel framework for proactive straggler mitigation in distributed environments.
• To enhance the efficiency and scalability of large-scale machine learning tasks.

Main Methods:

• Development of DPro-SM, a distributed framework utilizing Long Short-Term Memory (LSTM) networks.
• Implementation of LSTM for predicting worker completion times within the DDL framework.
• Proactive resource allocation strategies based on predicted completion times to mitigate stragglers.

Main Results:

• DPro-SM significantly reduces overall training time in distributed deep learning.
• The framework demonstrates improved scalability for large-scale machine learning tasks.
• Proactive straggler mitigation leads to enhanced training efficiency.

Conclusions:

• The proposed DPro-SM framework effectively tackles straggler node problems in DDL.
• LSTM-based prediction and proactive resource allocation are key to improving DDL performance.
• DPro-SM offers a promising solution for efficient and scalable distributed deep learning.