Parameter Efficient Fine-tuning of Transformer-based Masked Autoencoder Enhances Resource Constrained Neuroimage...

Area of Science:

• Artificial Intelligence
• Neuroimaging
• Medical Image Analysis

Background:

• Foundation models in AI are increasingly general-purpose, unlike specialized models.
• Transformer architectures are standard for foundation models across various data types.
• Parameter-Efficient Fine-Tuning (PEFT) methods are crucial for adapting models to specialized tasks, especially in medical imaging with limited data.

Purpose of the Study:

• To evaluate various PEFT methods on pre-trained vision transformers for neuroimaging tasks.
• To compare PEFT performance against full fine-tuning and training from scratch.
• To assess the efficiency and effectiveness of adapting foundation models for medical image analysis.

Main Methods:

• Pre-trained a vision encoder using a transformer-based masked autoencoder (MAE) framework on T1-weighted brain MRIs.
• Fine-tuned the pre-trained vision transformers using different PEFT methods, significantly reducing trainable parameters (as low as 0.04%).
• Evaluated performance on Alzheimer's disease (AD) and Parkinson's disease (PD) classification, and brain-age prediction.

Main Results:

• PEFT methods were competitive with or outperformed full fine-tuning and significantly outperformed training from scratch.
• PEFT methods improved Alzheimer's disease classification by 3% over full fine-tuning and 11% over 3D CNN with limited data.
• Smaller model sizes achieved competitive test performance, demonstrating efficiency.

Conclusions:

• PEFT methods offer an efficient and effective approach to adapt foundation models for neuroimaging tasks.
• These methods are valuable alternatives to training specialized models, especially given data limitations.
• The study highlights the potential of PEFT for diverse neuroimaging applications.