MS-DINO: Masked Self-Supervised Distributed Learning Using Vision Transformer.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Computer Vision

Background:

• Deep learning in medicine faces data scarcity and privacy issues.
• Federated learning offers distributed solutions but has communication/computation overhead and privacy risks.

Purpose of the Study:

• To propose a novel self-supervised distributed learning technique for Vision Transformers.
• To address communication overhead, privacy vulnerabilities, and computational burden in medical AI.

Main Methods:

• Developed MS-DINO, a self-supervised masked sampling distillation technique.
• Tailored MS-DINO for Vision Transformer architecture.
• Incorporated a modified encryption mechanism for enhanced privacy.

Main Results:

• MS-DINO eliminates the need for incessant communication.
• The method strengthens privacy safeguards.
• Significantly outperforms existing self-supervised distributed learning strategies and fine-tuned baselines.

Conclusions:

• MS-DINO offers an efficient and private self-supervised distributed learning approach for medical AI.
• The technique is particularly effective for Vision Transformer models.
• Provides a viable solution to data scarcity and privacy concerns in medical deep learning.