Client-Unbiased Skeletal Action Recognizer in Federated Learning.

Area of Science:

• Computer Vision
• Machine Learning
• Artificial Intelligence

Background:

• Edge devices generate extensive user data, raising privacy concerns with centralized processing.
• Federated Learning (FL) offers decentralized training but struggles with skeleton video analysis, failing to preserve motion dynamics and exhibiting client heterogeneity bias.

Purpose of the Study:

• To introduce CSAR (Client-Unbiased Skeletal Action Recognizer for Federated Learning), a novel framework designed to overcome limitations in FL for skeleton-based action recognition.
• To preserve crucial motion dynamics and mitigate classifier bias inherent in decentralized learning environments.

Main Methods:

• CSAR utilizes a Model Calibration Loss during client training to align representations and minimize data drift between clients and the server.
• Server-side processing involves Prototypical Gaussian Sampling for class-balanced spatiotemporal features, refined by a Motion-aware Differential Loss to capture kinematic properties.
• Knowledge Matching is employed for enhanced global model understanding and stabilization.

Main Results:

• CSAR enables the retraining of a globally debiased recognizer that achieves accuracy comparable to models trained on centralized data.
• Experiments demonstrate superior performance over state-of-the-art methods under both natural and label heterogeneity conditions.
• The proposed method effectively preserves motion dynamics and mitigates client bias in federated skeleton action recognition.

Conclusions:

• CSAR presents a robust solution for privacy-preserving skeleton action recognition using Federated Learning.
• The framework successfully addresses key challenges of motion dynamics preservation and classifier bias mitigation.
• CSAR offers a significant advancement for decentralized human action analysis, achieving high accuracy and generalization.