CreativeSeg: Semantic Segmentation of Creative Sketches.

Area of Science:

• Computer Vision
• Machine Learning
• Artificial Intelligence

Background:

• Sketch semantic segmentation struggles with creative, individualistic sketches due to distribution shifts.
• Existing models generalize poorly on highly creative sketches, limiting their real-world applicability.

Purpose of the Study:

• To develop a method that explicitly addresses the challenge of creativity in sketch semantic segmentation.
• To improve model generalization by learning creativity-agnostic sketch representations.

Main Methods:

• A learnable creativity estimator was developed to assign a creativity score to each sketch.
• CreativeSeg, a learning-to-learn framework, was introduced to leverage the creativity estimator.
• The framework learns creativity-agnostic representations for downstream semantic segmentation tasks.

Main Results:

• CreativeSeg demonstrated superior performance on the "Creative Birds" and "Creative Creatures" datasets.
• Empirical evidence confirmed the framework's effectiveness in handling creative sketches.
• A human study validated that the learned creativity score correlates with subjective human creativity.

Conclusions:

• CreativeSeg offers a robust solution for sketch semantic segmentation, particularly for imaginative and creative inputs.
• Disentangling creativity is a key factor in enhancing model generalization for sketch-based tasks.
• The proposed approach advances the field by effectively handling the variability introduced by human creativity.