CLIP-Driven Transformer for Weakly Supervised Object Localization.

Area of Science:

• Computer Vision
• Machine Learning
• Artificial Intelligence

Background:

• Weakly supervised object localization (WSOL) uses image-level labels for object localization.
• Current transformer-based WSOL methods often use category-agnostic attention, limiting localization accuracy.

Purpose of the Study:

• To develop a novel framework, CLIP-Driven TRansformer (CDTR), for accurate weakly supervised object localization.
• To improve object localization by learning category-aware representations.

Main Methods:

• Proposed a Category-aware Stimulation Module (CSM) to embed category biases into self-attention maps.
• Introduced an Object Constraint Module (OCM) for self-supervised refinement of object regions.
• Developed a Semantic Kernel Integrator (SKI) and a Semantic Boost Adapter (SBA) using the CLIP model to enhance object representations.

Main Results:

• The CDTR framework demonstrated superior performance on benchmark datasets like CUB-200-2011 and ILSVRC.
• Category-aware attention maps led to more accurate object localization compared to category-agnostic methods.

Conclusions:

• The proposed CDTR framework effectively addresses limitations in current WSOL methods.
• CDTR achieves state-of-the-art performance in weakly supervised object localization through category-aware learning and CLIP integration.