A federated learning with Large-Small Kernel Attention Network for image classification.

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Federated learning (FL) faces challenges with heterogeneous image data, impacting security, privacy, and performance.
• Existing FL frameworks struggle with complex image features and balancing collaboration with data security.

Purpose of the Study:

• Introduce FL-LSNet, a novel federated learning framework with a lightweight Large-Small Network (LSNet).
• Address data security, privacy, and performance degradation in collaborative image learning.

Main Methods:

• Developed FL-LSNet with a client-server architecture for decentralized preprocessing and data privacy.
• Integrated LSNet featuring Large Kernel Perceptrons (LKP) for global context and Small Kernel Attention (SKA) for local fusion.
• Implemented dynamic weight adjustment for long-tailed data and server-side aggregation.

Main Results:

• LSNet reduced computational overhead by 7% and improved feature representation by 19% compared to Swin Transformer and baseline models.
• FL-LSNet outperformed FedAvg and MOON on three datasets, achieving 84.32% to 98.92% accuracy.
• Ablation studies showed FedAvg-LSNet integration surpassed the baseline by 6.15%.

Conclusions:

• FL-LSNet offers a scalable solution for multi-stakeholder data collaboration in federated learning.
• Presents new insights into lightweight vertical adaptation of FL for public safety, agriculture, and medical diagnosis.