Group-Based Alternating Direction Method of Multipliers for Distributed Linear Classification.

Area of Science:

• Machine Learning
• Distributed Computing
• Optimization Algorithms

Background:

• The Alternating Direction Method of Multipliers (ADMM) is common in distributed machine learning but has slow convergence and high time costs.
• Existing distributed ADMM frameworks face challenges in achieving rapid convergence and robust global consensus.

Purpose of the Study:

• To introduce a novel Group-based ADMM (GADMM) for enhanced distributed linear classification.
• To address the limitations of traditional ADMM, specifically slow convergence and high computational expense.

Main Methods:

• GADMM utilizes a group layer to partition slave nodes, aggregating local variables into group variables.
• A weighted averaging method coordinates group variables to update the master node's global variable.
• Theoretical analysis establishes GADMM's convergence rate at O(1/k).

Main Results:

• GADMM demonstrates improved convergence speed over distributed ADMM without grouping.
• Experiments on LIBSVM datasets show GADMM reduces outer iterations, enhancing convergence and global consensus.
• Statistical tests confirm GADMM saves up to 30% time cost with minimal accuracy loss (<0.6%) compared to disADMM on large datasets.

Conclusions:

• GADMM offers a significant improvement for distributed linear classification tasks.
• The grouping strategy effectively accelerates convergence and reduces computational time.
• GADMM provides a practical and efficient solution for large-scale distributed machine learning problems.