Heterogeneity-aware Clustered Distributed Learning for Multi-source Data Analysis

Area of Science:

• Machine Learning
• Statistical Modeling
• Data Science

Background:

• Distributed data integration is common across fields like finance and omics.
• Privacy concerns often prevent raw data integration, necessitating distributed learning with summary statistics.
• Existing methods often assume homogeneous models across clients, limiting applicability to heterogeneous data.

Purpose of the Study:

• To develop a novel penalization approach for distributed learning that accommodates client clustering.
• To enable automatic clustering of clients into groups with shared models.
• To improve understanding of interconnections among distributed data sources and reduce model complexity.

Main Methods:

• A novel penalization approach incorporating group penalization for variable selection and fusion penalization for client clustering.
• Development of an effective Alternating Direction Method of Multipliers (ADMM) algorithm.
• Theoretical establishment of estimation, selection, and clustering consistency properties under mild conditions.

Main Results:

• The proposed method effectively clusters clients, assigning similar models within clusters and distinct models across clusters.
• Group and fusion penalization facilitate regularized estimation, variable selection, and automatic client clustering.
• The ADMM algorithm provides an efficient solution for the developed penalized objective function.

Conclusions:

• The novel penalization approach offers a superior method for distributed learning with clustered clients, addressing data heterogeneity.
• The technique enhances interpretability by revealing client interconnections and reduces computational burden through parameter reduction.
• Simulation studies and real-world data analysis validate the practical utility and effectiveness of the proposed distributed learning framework.