DH-MSVM: A hybrid algorithm for seeking quality support vectors in distributed learning.

Area of Science:

• Machine Learning
• Distributed Computing
• Data Science

Background:

• Data heterogeneity presents a significant challenge in distributed machine learning.
• Existing Distributed Support Vector Machines (DSVMs) struggle with identifying optimal support vectors across diverse data structures, hindering dynamic decision boundary adjustments.

Purpose of the Study:

• To propose a novel Distributed Hybrid Learning based on Support Vector Machine (DH-SVM) to address data heterogeneity.
• To enhance the DH-SVM algorithm with Markov sampling for improved computational efficiency and robustness (DH-MSVM).

Main Methods:

• Leveraging global pre-learning to capture data structure information for guiding local learning.
• Incorporating Markov sampling (DH-MSVM) to manage computational overhead in distributed learning.
• Theoretical derivation of generalization bounds using uniformly ergodic Markov chain samples.

Main Results:

• The proposed DH-SVM and DH-MSVM algorithms effectively identify higher-quality support vectors.
• Adaptive refinement of decision boundaries is achieved, improving model performance.
• Theoretical analysis confirms a fast learning rate, demonstrating robustness and scalability.

Conclusions:

• The DH-SVM approach, particularly with the DH-MSVM enhancement, offers a superior solution for handling data heterogeneity in distributed machine learning.
• The algorithms demonstrate improved performance and scalability validated through extensive empirical experiments on real-world datasets.