DPro-SM - 使用LSTM主动减轻滞后者的分布式框架
Aswathy Ravikumar1, Harini Sriraman1
1School of Computer Science and Engineering, VIT, Chennai, 600127, India.
Heliyon
|January 3, 2024
概括
分布式深度学习 (DDL) 由于滞后节点而面临训练延迟. 一个新的基于LSTM的框架,DPro-SM,主动地减轻了这些问题,大大减少了大规模机器学习的培训时间.
科学领域:
- 人工智能的人工智能
- 机器学习 机器学习
- 计算机科学 计算机科学
背景情况:
- 分布式深度学习 (DDL) 对大数据和高性能计算至关重要.
- 越来越多的数据量和网络复杂性加剧了DDL的挑战,导致高训练时间和精度降低.
- 滞后节点会造成通信瓶,大大延迟DDL培训过程.
研究的目的:
- 解决DDL.DL.中的通信上方和滞后节点问题.
- 为在分布式环境中主动缓解滞后者的新框架提出建议.
- 提高大型机器学习任务的效率和可扩展性.
主要方法:
- 开发DPro-SM,一个使用长短期内存 (LSTM) 网络的分布式框架.
- 在DDL框架内实施LSTM来预测工人完成时间.
- 基于预测完成时间的积极资源分配策略,以减轻滞后者.
主要成果:
- DPro-SM显著减少了分布式深度学习的整体培训时间.
- 该框架显示了大规模机器学习任务的可扩展性.
- 积极的滞后者缓解导致提高训练效率.
结论:
- 拟议的DPro-SM框架有效地解决了DDL中的滞后节点问题.
- 基于LSTM的预测和积极的资源配置是提高DDL性能的关键.
- DPro-SM为高效和可扩展的分布式深度学习提供了一个有前途的解决方案.
相关概念视频
02:37Restarting Stalled Replication Forks
5.8K
DNA replication is initiated at sites containing predefined DNA sequences known as origins of replication. DNA is unwound at these sites by the minichromosome maintenance (MCM) helicase and other factors such as Cdc45 and the associated GINS complex.The unwound single strands are protected by replication protein A (RPA) until DNA polymerase starts synthesizing DNA at the 5’ end of the strand in the same direction as the replication fork. To prevent the replication fork from falling apart,...
5.8K
01:19Protein Dynamics in Living Cells
2.1K
Different fluorescence-based techniques are used to study the protein dynamics in living cells. These techniques include FRAP, FRET, and PET.
Fluorescent recovery after photobleaching (FRAP) is a fluorescent-protein-based detection technique used to quantify protein movement rates within the cell. This method exposes a small portion of the cell to an intense laser beam. The laser beam causes permanent photobleaching of the fluorophore-tagged proteins in the exposed region. As the bleached...
Fluorescent recovery after photobleaching (FRAP) is a fluorescent-protein-based detection technique used to quantify protein movement rates within the cell. This method exposes a small portion of the cell to an intense laser beam. The laser beam causes permanent photobleaching of the fluorophore-tagged proteins in the exposed region. As the bleached...
2.1K