Unsupervised Pre-Training for Detection Transformers.

Area of Science:

• Computer Vision
• Machine Learning
• Artificial Intelligence

Background:

• Detection Transformer (DETR) achieves competitive object detection performance using a transformer encoder-decoder architecture.
• Training DETR from scratch requires extensive datasets and prolonged training schedules, limiting its practical application.
• Pre-training transformers has shown significant success in natural language processing, suggesting potential benefits for computer vision tasks.

Purpose of the Study:

• To develop an unsupervised pre-training method for DETR to reduce data and training time requirements.
• To introduce a novel pretext task, random query patch detection, for effective unsupervised pre-training of DETR.
• To enhance the fine-tuning capabilities of DETR for object detection, one-shot detection, and panoptic segmentation.

Main Methods:

• Proposed Unsupervised Pre-training DETR (UP-DETR) with a random query patch detection pretext task.
• Implemented multi-task learning and multi-query localization strategies during pre-training.
• Froze the CNN backbone during pre-training to balance classification and localization, and used an attention mask for multi-query localization.

Main Results:

• UP-DETR significantly improved DETR's performance across object detection, one-shot detection, and panoptic segmentation tasks.
• Achieved faster convergence and higher average precision compared to standard DETR training.
• Demonstrated the effectiveness of unsupervised pre-training for transformer-based object detection models.

Conclusions:

• UP-DETR offers a viable solution to the data and training time challenges associated with DETR.
• The proposed pretext task and training strategies effectively enhance DETR's capabilities.
• UP-DETR provides a unified framework for improving performance on diverse detection and segmentation tasks.