Related Experiment Video
Updated: Mar 14, 2026
07:12
Development of a Gaze-Contingent Display Framework Designed for Perceptual and Oculomotor Research with Simulated Central Vision Loss
Published on: April 11, 2025
1.0K
Efficient Training of Large Vision Models via Advanced Automated Progressive Learning
Summary
This study introduces automated progressive learning for Large Vision Models (LVMs), significantly reducing training costs and time. The AutoProg framework accelerates pre-training and fine-tuning of models like ViTs, diffusion, and autoregressive models with improved performance.
Area of Science:
- Computer Vision
- Machine Learning
- Artificial Intelligence
Background:
- Large Vision Models (LVMs) like Vision Transformers (ViTs), diffusion models, and visual autoregressive models require substantial computational resources, leading to high financial and environmental costs.
- Efficient training methods are crucial to mitigate the escalating resource demands of modern LVMs.
- Progressive learning, a strategy of gradually increasing model capacity during training, shows potential for enhancing efficiency.
Purpose of the Study:
- To develop and automate progressive learning strategies for efficient training of Large Vision Models (LVMs).
- To reduce the computational costs and time associated with pre-training, transfer learning, and fine-tuning of LVMs.
- To propose a comprehensive and scalable framework applicable to various LVM architectures.
Main Methods:
- Proposed AutoProg-One, an automated progressive learning scheme for Vision Transformer (ViT) pre-training, incorporating momentum growth (MoGrow) and one-shot growth schedule search.
- Extended the AutoProg framework with AutoProg-Zero, a zero-shot automated progressive learning method, eliminating the need for supernet training.
- Introduced a Unique Stage Identifier (SID) scheme to facilitate seamless network growth during training.
Main Results:
- AutoProg accelerated ViT pre-training on ImageNet by up to 1.85×.
- Accelerated fine-tuning of diffusion models and visual autoregressive models by up to 2.86× and 1.89×, respectively.
- Achieved comparable or superior performance metrics across all tested LVMs and training scenarios.
Conclusions:
- Automated progressive learning offers a robust and scalable solution for the efficient training of diverse Large Vision Models.
- The AutoProg framework significantly reduces training time and computational costs without compromising model performance.
- This approach has broad applicability for various computer vision tasks, promoting more sustainable AI development.
