Relation DETR+: Exploring Explicit Position Relation Prior for Dense Prediction

Area of Science:

• Computer Vision
• Deep Learning
• Artificial Intelligence

Background:

• Object detection models like DETR (DEtection TRansformer) suffer from slow convergence due to self-attention lacking structural input bias.
• Existing methods struggle to effectively integrate spatial relationships, limiting performance in dense prediction tasks.

Purpose of the Study:

• To propose a novel scheme, Relation-DETR+, for enhancing the convergence and performance of DETR-based models.
• To address the slow convergence issue by incorporating explicit position relation priors into the attention mechanism.
• To develop a unified framework capable of handling multiple dense prediction tasks including object detection, semantic segmentation, instance segmentation, and panoptic segmentation.

Main Methods:

• Incorporation of position relation prior as attention bias to augment object detection.
• Introduction of an encoder for constructing position relation embeddings for progressive attention refinement.
• Extension of the DETR pipeline into a contrastive relation pipeline to manage prediction conflicts.
• Alleviation of pattern collapse in multi-layer relations via explicit layer-wise encoding and gated relation modulation.

Main Results:

• Relation-DETR+ demonstrates superior performance and learning efficiency compared to state-of-the-art methods like DINO and Mask-DINO under similar training schedules.
• The proposed relation encoder acts as a plug-and-play component, improving various DETR-like methods.
• Extensive experiments on diverse datasets validate the effectiveness of the approach for unified dense prediction tasks.
• Introduction of SA-Det-100k, a large-scale class-agnostic detection dataset, highlighting the potential of explicit position relations for universal object detection.

Conclusions:

• The proposed Relation-DETR+ framework effectively enhances DETR performance by integrating explicit position relations, leading to improved convergence and efficiency.
• The approach offers a unified solution for multiple dense prediction tasks within a single framework.
• The relation encoder is a versatile component applicable to a wide range of DETR-based architectures, advancing the field of computer vision.