Evaluation for Weakly Supervised Object Localization: Protocol, Metrics, and Datasets.

Area of Science:

• Computer Vision
• Machine Learning

Background:

• Weakly-supervised object localization (WSOL) trains models using only image-level labels, avoiding costly bounding box annotations.
• Class Activation Mapping (CAM) is a foundational technique, but subsequent methods often require full supervision for validation, contradicting WSOL principles.

Purpose of the Study:

• To address the ill-posed nature of WSOL with limited supervision.
• To propose a novel evaluation protocol for WSOL that uses a small, held-out set for full supervision.
• To re-evaluate existing WSOL methods under this new protocol.

Main Methods:

• Introduced a new WSOL evaluation protocol restricting full supervision to a small, separate validation set.
• Compared five recent WSOL methods against the Class Activation Mapping (CAM) baseline.
• Evaluated WSOL methods against a few-shot learning baseline using full supervision during validation.

Main Results:

• Under the proposed protocol, recent WSOL methods showed no significant improvement over the basic CAM baseline.
• Existing WSOL methods did not outperform the few-shot learning baseline, which utilizes full supervision for validation.
• The study highlights limitations in current WSOL evaluation and performance.

Conclusions:

• The WSOL task, as commonly evaluated, may be ill-posed due to reliance on prohibited full supervision for validation.
• A more rigorous evaluation protocol is necessary to accurately assess WSOL advancements.
• Future research should focus on developing methods that genuinely perform well under strict weak supervision constraints.