ViT-MVT: A Unified Vision Transformer Network for Multiple Vision Tasks.

Area of Science:

• Computer Vision
• Deep Learning
• Artificial Intelligence

Background:

• Traditional deep learning models often require separate networks for each vision task, leading to high storage requirements.
• Vision Transformer (ViT) architectures have shown promise but often lack efficiency when applied to multiple tasks simultaneously.

Purpose of the Study:

• To develop a storage-efficient, unified network capable of concurrently learning multiple mainstream vision tasks.
• To challenge the conventional approach of using image classification backbones for diverse visual tasks.

Main Methods:

• Developed ViT-MVT, a framework built on a non-hierarchical ViT, incorporating multi-head self-attention (MHSE), local perception unit (LPU), and locality feed-forward network (locality FFN).
• Implemented a layer-adaptive sharing technique to balance shared and task-specific parameters for optimal storage and performance.
• Utilized a joint-task evolutionary search algorithm to discover an optimal backbone under model size constraints.

Main Results:

• ViT-MVT achieved state-of-the-art performance across multiple vision tasks while significantly reducing total storage costs.
• Demonstrated the framework's capability for incremental learning, enabling generalization to new tasks without performance degradation on previously trained tasks.

Conclusions:

• ViT-MVT offers a highly efficient and effective solution for multi-task learning in computer vision.
• The proposed layer-adaptive sharing and evolutionary search methods provide a novel approach to designing versatile vision backbones.