DHS-AE: A Distributed Support Vector Machine With Adaptive Regularization Parameters for Different Data...

Area of Science:

• Machine Learning
• Distributed Systems
• Data Science

Background:

• Distributed machine learning faces challenges with varying data distributions across nodes.
• Existing methods struggle with autonomous parameter adjustment for dynamic data, leading to rigid global boundaries and poor local adaptation.

Purpose of the Study:

• To propose a novel distributed hybrid support vector machine (SVM) model, termed DHS-AE, capable of adaptive ensemble selection of regularization parameters.
• To enable real-time adjustment of decision boundaries in response to data distribution shifts.

Main Methods:

• The DHS-AE model leverages data structure information to partition the data space, identifying distinct data distribution characteristics.
• Support vector machines (SVMs) with adaptively determined regularization parameters are employed within local subspaces.
• Theoretical generalization bounds are established using covering numbers.

Main Results:

• The proposed DHS-AE model demonstrates fast convergence speed and consistency.
• The method effectively reduces computational overhead by enabling localized decision boundary adjustments.
• Empirical validation on numerous real-world datasets confirms the model's superior performance.

Conclusions:

• The DHS-AE model provides an effective solution for distributed machine learning with heterogeneous data distributions.
• Adaptive regularization parameter selection enhances local adaptation and model flexibility.
• The theoretical and practical results highlight the potential of DHS-AE for robust distributed learning.