Distributed Online Learning With Multiple Kernels.

Area of Science:

• Machine Learning
• Distributed Systems
• Optimization

Background:

• Decentralized learning models are crucial for handling large-scale, distributed data.
• Online learning scenarios require continuous adaptation to streaming data.
• Existing federated learning approaches often rely on centralized coordination.

Purpose of the Study:

• To develop a fully decentralized online learning framework for nonlinear function approximation.
• To ensure privacy by keeping data localized at individual learners.
• To achieve theoretical performance guarantees comparable to centralized methods.

Main Methods:

• Proposed a novel framework named DOMKL (Distributed Online Multiple Kernel Learning).
• Integrated principles from online alternating direction method of multipliers (ADMM) and distributed Hedge algorithm.
• Theoretically analyzed the regret bound for the proposed algorithm.

Main Results:

• DOMKL achieves an optimal sublinear regret of O(√T) over T time slots.
• Demonstrated that DOMKL matches the asymptotic performance of state-of-the-art centralized approaches.
• Empirically validated DOMKL's effectiveness on real-world online regression and time-series prediction tasks.

Conclusions:

• DOMKL provides an effective solution for fully decentralized online learning of nonlinear functions.
• The framework successfully balances performance with data privacy by maintaining local data.
• DOMKL offers a promising alternative to centralized methods in distributed learning settings.