Learning a Dictionary of Shape Epitomes with Applications to Image Labeling

Area of Science:

• Computer Vision
• Machine Learning
• Image Processing

Background:

• Traditional image representations like superpixels or pixels in Conditional Random Field (CRF) models can lead to irreversible early decisions.
• Existing methods may not optimally capture local edge structures and their variations (shifts, rotations) in natural images.

Purpose of the Study:

• To propose a novel image representation method using a dictionary of shape epitomes.
• To apply shape epitomes to image labeling tasks, offering an alternative to pixel or superpixel representations.
• To develop hierarchical CRFs that efficiently capture both local and global class co-occurrence properties.

Main Methods:

• Learned a compact dictionary of shape epitomes unsupervised from groundtruth edges, encoding local image edge structure with hidden variables for shifts and rotations.
• Applied shape epitomes to image labeling by integrating them into Conditional Random Field (CRF) models as an alternative representation.
• Developed hierarchical CRFs that avoid early irreversible decisions, unlike superpixel methods, by encoding image patch shapes with shape epitomes.

Main Results:

• Demonstrated the effectiveness of shape epitomes in image labeling through quantitative and qualitative experiments.
• Showcased the ability of the shape epitome dictionary to capture typical edge shapes in natural images.
• Validated the approach on standard datasets: MSRC-21 and Stanford Background.

Conclusions:

• Shape epitomes provide a robust and flexible image representation for tasks like image labeling.
• Hierarchical CRFs utilizing shape epitomes offer an improved approach to image analysis by capturing complex spatial relationships.
• The method avoids limitations of early decision-making inherent in superpixel-based CRFs.