Multi-Task Distributed Learning Using Vision Transformer With Random Patch Permutation.

Area of Science:

• Artificial Intelligence in Health Research
• Distributed Machine Learning
• Medical Imaging Analysis

Background:

• Data availability limitations hinder AI application in health research.
• Federated Learning (FL) and Split Learning (SL) address data challenges but have distinct drawbacks.
• Federated Split Task-Agnostic (FeSTA) learning combines FL and SL using Vision Transformers (ViT) but incurs high communication overhead.

Purpose of the Study:

• To propose a novel multi-task distributed learning method using ViT.
• To improve upon FeSTA by reducing communication overhead and enhancing performance.
• To ensure privacy preservation in collaborative health research.

Main Methods:

• Introduced p-FeSTA: a multi-task distributed learning approach using ViT with random patch permutation.
• Replaced the CNN-based head in FeSTA with a simple patch embedder and random permutation.
• Evaluated performance, communication efficiency, and privacy preservation.

Main Results:

• p-FeSTA significantly enhances multi-task collaboration benefits.
• The method improves communication efficiency compared to existing approaches.
• p-FeSTA demonstrates effective privacy preservation without sacrificing performance.
• Experimental results validate the proposed method's advantages.

Conclusions:

• p-FeSTA offers a practical solution for multi-task distributed learning in medical imaging.
• The approach effectively balances performance, communication efficiency, and privacy.
• This work paves the way for more widespread adoption of distributed AI in healthcare.