Heterogeneous Collaborative Learning for Personalized Healthcare Analytics via Messenger Distillation

Area of Science:

• Healthcare technology
• Distributed artificial intelligence
• Edge computing

Background:

• Healthcare Internet-of-Things (IoT) enables personalized medicine via edge devices.
• Data sparsity on individual devices necessitates cross-device collaboration for robust AI.
• Conventional collaborative learning requires homogeneous models, which is impractical for heterogeneous edge devices with varying architectures and asynchronous participation.

Purpose of the Study:

• To propose a novel framework, Similarity-Quality-based Messenger Distillation (SQMD), for heterogeneous and asynchronous on-device healthcare analytics.
• To enable knowledge distillation among diverse edge devices without requiring identical model architectures.
• To enhance the personalization and reliability of collaborative healthcare analytics in asynchronous settings.

Main Methods:

• SQMD utilizes a preloaded reference dataset for knowledge distillation via 'messengers' (soft labels).
• Messengers transmit auxiliary information to assess client model similarity and quality.
• A central server dynamically manages a collaboration graph based on messenger-derived metrics to optimize asynchronous learning.

Main Results:

• SQMD successfully enables knowledge distillation across heterogeneous on-device models.
• The framework effectively handles asynchronous client participation.
• Experimental validation on three real-life datasets demonstrates superior performance compared to existing methods.

Conclusions:

• SQMD offers a robust solution for on-device healthcare analytics in heterogeneous and asynchronous environments.
• The proposed messenger distillation and dynamic collaboration graph significantly improve personalization and reliability.
• This framework advances the application of distributed AI in personalized healthcare.