HIRI-ViT: Scaling Vision Transformer With High Resolution Inputs.

Area of Science:

• Computer Vision
• Deep Learning Architectures
• Image Recognition

Background:

• Hybrid deep models combining Vision Transformer (ViT) and Convolutional Neural Networks (CNNs) are effective for vision tasks.
• Increasing input resolution enhances model capacity but leads to computationally expensive, quadratically scaling costs.
• Existing hybrid backbones struggle with efficiency when handling high-resolution inputs.

Purpose of the Study:

• To develop a novel hybrid deep backbone, HIRI-ViT, specifically designed for efficient processing of high-resolution inputs.
• To improve computational efficiency and model performance in vision tasks by addressing the quadratic scaling issue of high-resolution inputs.
• To establish a new state-of-the-art in image recognition and dense prediction tasks with a cost-efficient architecture.

Main Methods:

• Proposed HIRI-ViT, a five-stage Vision Transformer (ViT) backbone tailored for high-resolution inputs.
• Implemented a novel approach decomposing CNN operations into two parallel branches: one high-resolution with fewer convolutions, and one low-resolution with more convolutions after down-sampling.
• Evaluated HIRI-ViT on ImageNet-1K (recognition) and COCO, ADE20K (dense prediction) datasets.

Main Results:

• HIRI-ViT demonstrated superior performance on both image recognition and dense prediction tasks.
• Achieved the best published Top-1 accuracy of 84.3% on ImageNet with 448x448 inputs, surpassing previous methods.
• Under comparable computational cost (~5.0 GFLOPs), HIRI-ViT significantly improved accuracy compared to existing models like iFormer-S.

Conclusions:

• HIRI-ViT offers a highly effective and computationally efficient solution for processing high-resolution images in deep learning.
• The proposed parallel CNN branch decomposition effectively balances feature resolution and computational cost.
• HIRI-ViT sets a new benchmark for performance in vision tasks, particularly for high-resolution image analysis.