Distribution bias embedding tuning of vision transformer for remote sensing object detection

The expanded availability of remote sensing data in recent years has yielded an increasing demand for advanced object detection methods to support Earth observation applications. Vision Transformers (ViTs) have recently emerged as a powerful backbone in the field of visual recognition, achieving exceptional performance in various tasks. However, adapting the pretrained ViTs to specific downstream tasks, particularly in remote sensing, represents a challenge due to the different distribution between the pretraining and downstream task datasets. To address this challenge, this paper introduces the distribution bias embedding-based tuning (DBET) method. DBET integrates a bias embedding token, which dynamically adjusts the feature representations to better align the distribution bias between the pretraining and downstream task datasets. DBET also reduces the number of parameters that require fine-tuning, thus significantly improving computational efficiency and reducing training time while maintaining the original structure unchanged. DBET is verified by experimental evaluations on benchmark datasets, including the DOTA-v1.0, RarePlanes, and EAGLE datasets. The results demonstrate that DBET can not only reduce the number of trainable parameters and accelerate the fine-tuning process but can also significantly improve accuracy, particularly for tasks with a limited number of categories, providing a scalable solution for remote sensing object detection. The DBET method achieved a recognition accuracy of 56.94% on the EAGLE dataset, 80.56% on the five-class RarePlanes dataset, and 70.68% on the DOTA-v1.0 dataset. Moreover, the DBET method reduced the average training time to 58% of full fine-tuning method.