Related Experiment Video
Updated: Jun 16, 2026
04:48
Swin-PSAxialNet: An Efficient Multi-Organ Segmentation Technique
Published on: July 5, 2024
Convolutional low-rank adaptation for efficient semantic segmentation in vision transformers
Srihari Srinivasan1, Morvin Prajapati1, Ananthakrishna Thalengala2
1Manipal Institute of Technology, Manipal Academy of Higher Education, Manipal, India.
Scientific Reports
|June 13, 2026
Summary
This study introduces Conv-LoRA, a method to efficiently adapt Vision Transformers for dual tasks like depth estimation and human segmentation. It achieves high accuracy with minimal parameters, reducing computational costs.
Area of Science:
- Computer Vision
- Deep Learning
- Artificial Intelligence
Background:
- Vision Transformers (ViTs) excel in computer vision but are computationally expensive for dense prediction tasks like semantic segmentation.
- Adapting large pre-trained models for specialized tasks requires significant computational resources.
Purpose of the Study:
- To develop a computationally efficient method for adapting Vision Transformers (ViTs) for dual-task learning, specifically monocular depth estimation and human semantic segmentation.
- To integrate low-rank adaptation techniques into ViTs to reduce the number of trainable parameters and computational overhead.
Main Methods:
- Proposed a novel dual-task architectural application using the LyCORIS Low-Rank Adaptation for Convolutions (LyCORIS LoCon) framework.
- Introduced learnable low-rank convolutional modules into pre-trained ViTs, specifically applied to the Depth Anything V2 (DAV2) model.
- Augmented the DAV2 decoder for dual-task outputs (depth estimation and human segmentation) with minimal parameter injection (150K trainable parameters).
Main Results:
- Achieved competitive semantic segmentation performance comparable to state-of-the-art models (SAM, MaskFormer, SegFormer, Mask2Former).
- Obtained an mAP of 89.69% and mIoU of 79.17% for human segmentation on filtered COCO and ImageNet subsets.
- Preserved the original depth prediction accuracy of the base DAV2 model while enhancing task-specific learning with minimal computational overhead.
Conclusions:
- Conv-LoRA offers an efficient and effective approach for adapting pre-trained Vision Transformers for multi-task learning in dense prediction.
- The proposed method significantly reduces adaptation costs without compromising performance, making advanced computer vision models more accessible.
- This work demonstrates the potential of low-rank adaptation techniques for enhancing the versatility and efficiency of large-scale vision models.