Learning Structural Representations via Dynamic Object Landmarks Discovery for Sketch Recognition and Retrieval.

Area of Science:

• Computer Vision
• Machine Learning
• Artificial Intelligence

Background:

• Deep convolutional neural networks (CNNs) are standard for sketch analysis but struggle with structural representations and visual variations.
• Fixed computational modes in CNNs limit their ability to capture human-perceptible object structures in sketches.
• Large scale visual variations in sketches pose challenges for developing generalized feature representations.

Purpose of the Study:

• To address the limitations of fixed computational modes in CNNs for sketch feature extraction.
• To propose a novel architecture for dynamically discovering object landmarks and learning discriminative structural representations without extra supervision.
• To enhance sketch classification and retrieval performance by capturing category-specific features.

Main Methods:

• A novel architecture comprising a landmark discovering module and a category-aware representation learning module.
• A structure-aware offset layer for dynamic landmark localization, optimized using category labels.
• A diversity branch for extracting global discriminative features and a multi-task loss function for end-to-end training.

Main Results:

• The proposed model effectively discovers object landmarks and learns discriminative structural representations.
• Experimental results on TU-Berlin and Sketchy datasets demonstrate superior performance in sketch classification and retrieval.
• The fusion of predictions with varying numbers of landmarks at test time enhances final results.

Conclusions:

• The novel architecture overcomes limitations of fixed CNN computational modes in sketch analysis.
• Dynamic landmark discovery and structural representation learning lead to improved sketch classification and retrieval.
• The model offers a more generalized and effective approach to understanding sketch data.