DN-DETR: Accelerate DETR Training by Introducing Query DeNoising.

Area of Science:

• Computer Vision
• Machine Learning
• Deep Learning

Background:

• DETR (DEtection TRansformer) and similar methods suffer from slow convergence due to unstable bipartite graph matching in early training stages.
• This instability leads to inconsistent optimization objectives, hindering efficient model training.

Purpose of the Study:

• To develop a novel denoising training method to accelerate DETR training.
• To address the slow convergence issue in DETR-like object detection models.
• To provide a deeper understanding of the convergence challenges in transformer-based detectors.

Main Methods:

• Introduced a denoising training strategy that supplements the Hungarian loss.
• Fed noisy ground truth bounding boxes into the Transformer decoder to train reconstruction of original boxes.
• This method is designed to be universally applicable to DETR-like architectures with minimal code changes.

Main Results:

• The proposed DN-DETR method achieved a +1.9 AP improvement under identical settings.
• Achieved 46.0 AP and 49.5 AP with ResNet-50 backbone after 12 and 50 epochs, respectively.
• Demonstrated comparable performance to baseline methods using 50% fewer training epochs.

Conclusions:

• Denoising training effectively reduces bipartite graph matching difficulty, leading to faster convergence.
• The method shows significant improvements in object detection accuracy and training efficiency.
• Validated the effectiveness of denoising training across various architectures, including CNN-based detectors and segmentation models.