Optimizing ViT-LoRA: A Memory-Efficient Approach for Fine-Tuning.

Area of Science:

• Medical Imaging
• Computer Vision
• Deep Learning

Background:

• Vision Transformers (ViT) are powerful for medical imaging but demand high computational resources.
• Existing ViT models present challenges in memory usage and training time for practical applications.

Purpose of the Study:

• To introduce ViT-LoRA, a parameter-efficient adaptation of ViTs for medical imaging.
• To address the computational and memory limitations of standard ViTs.

Main Methods:

• Implemented Low-Rank Adaptation (LoRA) within the ViT architecture.
• Reduced trainable parameters to 2.104 million while maintaining an overall parameter size of 137.07 million.
• Evaluated performance on medical imaging tasks, including the Lung Infection dataset.

Main Results:

• ViT-LoRA achieved a testing accuracy of 98.49%, outperforming the baseline ViT (96.60%) and ResNet models.
• Significantly reduced memory usage from 1568.62 MB to 24.08 MB and model size from 1500 MB to 539.2 MB.
• Decreased training time by 53.5% (400.32 seconds vs. 862.23 seconds).

Conclusions:

• ViT-LoRA provides a highly efficient and effective solution for medical imaging tasks using Vision Transformers.
• The method demonstrates superior performance, reduced resource requirements, and faster training compared to baseline ViT and ResNet models.
• ViT-LoRA shows consistent outperformance, particularly on the Lung Infection dataset.