Knowledge-Guided Label Distribution Calibration for Federated Affective Computing

The federated learning (FL) paradigm can significantly solve the rising public concern about data privacy in affective computing. However, conventional FL methods perform poorly due to the uniqueness of the task, as the personalized emotion data vary from client to client. To resolve the privacy-utility paradox, this work proposes a framework that largely improves federated affective computing (FAC) via calibrating the global feature space and communicating privacy-agnostic auxiliary information. The framework consists of two components: first, an emotion hemisphere (EH) representation structure is proposed, which utilizes emotional prior knowledge to unify the emotion global feature space of different clients. Second, the server uses the normalized parameter importance matrix to guide the model aggregation. It retains crucial parameters for individual local models, thereby alleviating the slow convergence problem in the global model caused by the skewed label distribution. The proposed framework yields significant performance gains, and extensive experiments on three emotion datasets demonstrate the effectiveness and the practicality of our approach.