TS-CAM: Token Semantic Coupled Attention Map for Weakly Supervised Object Localization.

Area of Science:

• Computer Vision
• Machine Learning
• Artificial Intelligence

Background:

• Weakly supervised object localization (WSOL) using only image category labels is challenging.
• Convolutional Neural Networks (CNNs) often fail to localize the full object extent, focusing instead on discriminative parts due to difficulties in capturing long-range semantic dependencies.

Purpose of the Study:

• To address the limitations of CNNs in WSOL by leveraging the Vision Transformer (ViT).
• To propose a novel method, Token Semantic Coupled Attention Map (TS-CAM), for improved object localization.

Main Methods:

• Introduced the Vision Transformer to WSOL to capture long-range semantic dependencies via self-attention.
• Developed TS-CAM, which decomposes class-aware semantics and couples them with attention maps for semantic-aware activation.
• Employed spatial embedding by partitioning images into patch tokens and reallocating category semantics to these tokens for improved long-distance feature capture.

Main Results:

• TS-CAM significantly outperformed CNN-based methods, achieving 11.6% and 28.9% improvements on ILSVRC and CUB-200-2011 datasets, respectively.
• Demonstrated state-of-the-art performance in WSOL.
• Showcased superior performance for multicategory object localization on the Pascal VOC dataset.

Conclusions:

• The Vision Transformer, through the TS-CAM method, effectively captures long-range semantic dependencies for robust object localization.
• TS-CAM overcomes the limitations of CNNs in WSOL, providing accurate localization of full object extents.
• The proposed approach offers significant advancements in both single- and multicategory weakly supervised object localization.