Fine-Tuned Segment Anything Model with Low-Rank Adaptation for Chest X-Ray Images

Area of Science:

• Medical Imaging
• Artificial Intelligence
• Computer Vision

Background:

• Chest X-ray (CXR) analysis is crucial for diagnosing respiratory conditions.
• Accurate segmentation of lung structures in CXRs is essential for quantitative analysis.
• Existing segmentation models may require extensive training data and computational resources.

Purpose of the Study:

• To evaluate the performance of the Segment Anything Model (SAM) for CXR segmentation.
• To investigate the effectiveness of low-rank adaptation (LoRA) for fine-tuning SAM in the medical domain.
• To compare fine-tuned SAM with zero-shot SAM and traditional Convolutional Neural Network (CNN) models.

Main Methods:

• Three SAM approaches were tested: zero-shot (coordinate and bounding box prompts) and fine-tuned using LoRA.
• U-Net and DeepLabv3+ CNNs were trained to provide initial segmentation prompts for SAM.
• LoRA was applied to SAM by adding lightweight adapters to Transformer blocks, freezing most parameters.

Main Results:

• Fine-tuned SAM with LoRA demonstrated superior segmentation accuracy over zero-shot SAM methods.
• The LoRA-based approach also showed improved efficiency compared to baseline CNNs.
• Segmentation performance was evaluated on a CXR dataset including COVID-19, viral pneumonia, and normal cases.

Conclusions:

• Combining LoRA with SAM offers a promising strategy for efficient and accurate medical image segmentation.
• This approach preserves SAM's pre-trained knowledge while reducing computational demands.
• The fine-tuned SAM with LoRA shows potential for practical clinical applications in CXR analysis.