Towards Better Caption Supervision for Object Detection.

Area of Science:

• Computer Vision
• Machine Learning
• Artificial Intelligence

Background:

• Training high-performance object detectors typically requires costly bounding box annotations.
• Existing methods using image captions for supervision yield suboptimal results due to noisy and imprecise location information in captions.

Purpose of the Study:

• To develop a visual analysis method that synergistically integrates caption supervision with object detection.
• To enhance the performance of object detectors by leveraging readily available image captions.

Main Methods:

• Object labels are extracted from image captions to initially train object detectors.
• Detected objects are then fed back into the caption supervision process for iterative refinement.
• A node-link-based set visualization and a multi-type relational co-clustering algorithm are employed to analyze label-image relationships.

Main Results:

• The integrated method demonstrates improved performance in object detection tasks.
• The co-clustering algorithm effectively groups labels and images based on their representations and interrelations.
• Quantitative evaluations and a case study confirm the method's efficiency and effectiveness.

Conclusions:

• The proposed visual analysis method offers a robust solution for improving object detection using noisy caption data.
• The synergistic integration of caption supervision and object detection leads to mutually beneficial enhancements.
• The developed visualization and co-clustering techniques provide valuable insights into label-image relationships.