Weakly-Supervised Image Annotation and Segmentation with Objects and Attributes.

Area of Science:

• Computer Vision
• Machine Learning
• Artificial Intelligence

Background:

• Modeling complex visual scenes is challenging due to the scarcity of precisely annotated data.
• Weakly labeled images, common on media-sharing platforms, offer a vast but unstructured data source.
• Existing methods often struggle to learn object-attribute relationships from limited supervision.

Purpose of the Study:

• To develop a non-parametric Bayesian model capable of learning from weakly labeled images.
• To enable joint modeling of object appearance, attributes, and their associations.
• To address multiple computer vision tasks including object recognition, attribute prediction, and segmentation.

Main Methods:

• A novel Weakly Supervised Markov Random Field Stacked Indian Buffet Process (WS-MRF-SIBP) was developed.
• The model treats objects and attributes as latent factors, capturing correlations within and across superpixels.
• Learning occurs from image-level annotations without explicit location or association information.

Main Results:

• The proposed WS-MRF-SIBP model significantly outperforms other weakly supervised approaches.
• Performance is comparable to strongly supervised models on various tasks.
• Demonstrated effectiveness in semantic segmentation, automatic image annotation, and retrieval.

Conclusions:

• Weakly supervised learning can achieve high performance in complex visual scene understanding.
• The WS-MRF-SIBP model provides a robust framework for learning from abundant, weakly labeled image data.
• This approach offers a scalable solution for tasks requiring object and attribute understanding.