Multi-Task Federated Split Learning Across Multi-Modal Data with Privacy Preservation.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Data Security

Background:

• Federated learning (FL) faces challenges with multi-modal data and multi-task learning, particularly in privacy-sensitive applications like intelligent connected vehicles.
• Existing FL schemes struggle with communication overhead and computational demands when handling complex, multi-modal datasets.

Purpose of the Study:

• To propose a novel privacy-preserving scheme for multi-task federated split learning across multi-modal data (MTFSLaMM).
• To address the limitations of traditional FL in handling multi-modal data and ensuring robust privacy protection.

Main Methods:

• Leveraging split learning to partition models between clients and servers, reducing client-side computational burden.
• Integrating differential privacy for intermediate data protection and homomorphic encryption for client model security.
• Employing an optimized attention mechanism guided by mutual information for efficient multi-modal data fusion.

Main Results:

• The proposed MTFSLaMM scheme effectively handles multi-modal data and multi-task learning challenges.
• Demonstrated robust privacy protection through differential privacy and homomorphic encryption.
• Achieved significant performance improvements: 15.3% in BLEU-4 and 11.8% in CIDEr scores compared to baseline methods.

Conclusions:

• MTFSLaMM offers an effective solution for privacy-preserving multi-task learning on multi-modal data.
• The scheme enhances data fusion efficiency and reduces computational overhead, making it suitable for resource-constrained environments like intelligent vehicles.