DRTN: Dual Relation Transformer Network with feature erasure and contrastive learning for multi-label image...

Area of Science:

• Computer Vision
• Artificial Intelligence
• Machine Learning

Background:

• Multi-label image classification (MLIC) aims to identify multiple objects in an image.
• Existing Transformer-based methods flatten 2D feature maps, losing spatial information.
• Current attention models may overlook potentially useful features for MLIC.

Purpose of the Study:

• To introduce a novel Dual Relation Transformer Network (DRTN) for end-to-end multi-label image classification.
• To address the loss of spatial information in Transformer-based MLIC methods.
• To enhance the learning of discriminative and comprehensive features for MLIC.

Main Methods:

• A grid aggregation scheme generates pseudo-region features to recover spatial information.
• A dual relation enhancement (DRE) module captures object correlations using dual visual features.
• A feature enhancement and erasure (FEE) module mines discriminative and potential features.
• A contrastive learning (CL) module refines feature learning by distinguishing foreground and background features.

Main Results:

• The DRTN method achieves superior performance compared to current MLIC models.
• Experiments were conducted on challenging benchmarks: MS-COCO 2014, PASCAL VOC 2007, and NUS-WIDE.
• The proposed modules effectively compensate for lost spatial information and enhance feature discrimination.

Conclusions:

• The DRTN offers a robust solution for multi-label image classification.
• The integration of grid aggregation, DRE, FEE, and CL modules leads to comprehensive feature learning.
• DRTN demonstrates significant improvements on established MLIC benchmarks.