FACT: A Simple and Efficient Framework for Active Finetuning.

Area of Science:

• Computer Science
• Machine Learning
• Artificial Intelligence

Background:

• Active finetuning aims to enhance pretrained models for specific tasks using selected data.
• Prior work focused on data selection, often using full finetuning, which can distort features and increase overfitting, especially with large models and limited data.

Purpose of the Study:

• To systematically explore finetuning methodologies within active learning for improved model adaptation.
• To introduce and evaluate the FACT (Finetuning-Aware Active learning Framework) framework for active finetuning scenarios.

Main Methods:

• The study outlines the FiAF (Finetuning-Aware Active learning) task.
• A three-phase hierarchical finetuning framework, FACT, was proposed for efficiency and simplicity.
• Experiments involved diverse datasets (classic, imbalanced, fine-grained), various pretrained architectures (ConvNeXt, ViT, ViL), and frozen feature augmentation (FroFA) strategies.

Main Results:

• The FACT framework demonstrated significant improvements in performance, generalization, and robustness across various datasets and architectures.
• Remarkable gains exceeding 20% were observed on ViT models for CIFAR10, CIFAR100, and ImageNet-1k under low sampling ratios.
• The approach established new state-of-the-art performance while maintaining parameter efficiency.

Conclusions:

• The proposed FACT framework effectively addresses the limitations of full finetuning in active learning by integrating finetuning strategies.
• This systematic approach offers a robust and efficient solution for active finetuning, particularly beneficial in low-data regimes.
• The findings highlight the importance of considering finetuning methodologies alongside data selection for optimal model adaptation.