MCTformer+: Multi-Class Token Transformer for Weakly Supervised Semantic Segmentation.

Area of Science:

• Computer Vision
• Machine Learning
• Artificial Intelligence

Background:

• Weakly supervised semantic segmentation (WSSS) traditionally struggles with precise object localization.
• Standard vision transformers generate class-agnostic localization maps, limiting their discriminative power.
• Existing methods often lack the ability to differentiate between object classes effectively.

Purpose of the Study:

• To propose a novel transformer-based framework for generating accurate class-specific object localization maps in WSSS.
• To enhance the transformer's capability for class-discriminative object localization.
• To improve the performance of WSSS by leveraging class-specific attention mechanisms.

Main Methods:

• Introduced the Multi-Class Token transformer with multiple class tokens for class-aware interactions.
• Implemented a class-aware training strategy for one-to-one correspondence between class tokens and labels.
• Developed a Contrastive-Class-Token (CCT) module to learn discriminative class tokens.
• Utilized patch-level pairwise affinity from transformer attention for localization map refinement.

Main Results:

• The framework effectively generates class-discriminative object localization maps.
• Significant performance improvements were observed in WSSS on PASCAL VOC 2012 and MS COCO 2014 datasets.
• The proposed method complements existing Class Activation Mapping (CAM) techniques.

Conclusions:

• The class token plays a crucial role in advancing WSSS.
• The Multi-Class Token transformer offers a promising direction for improving object localization in WSSS.
• The proposed framework demonstrates the effectiveness of class-specific attention for discriminative localization.