Multigraph Transformer for Free-Hand Sketch Recognition

Area of Science:

• Computer Vision
• Machine Learning
• Artificial Intelligence

Background:

• Sketch representation is challenging due to signal sparsity and high abstraction.
• Existing methods use Convolutional Neural Networks (CNNs) for static features or Recurrent Neural Networks (RNNs) for temporal sequences.

Purpose of the Study:

• To propose a novel method for learning meaningful representations of free-hand sketches.
• To introduce a new graph neural network (GNN) model for sketch recognition.

Main Methods:

• Representing sketches as multiple sparsely connected graphs.
• Designing a novel multigraph transformer (MGT) model to learn from these graph representations.
• Capturing global and local geometric structures along with temporal information.

Main Results:

• The MGT achieved a small recognition gap compared to CNN upper bounds (72.80% vs. 74.22%) on 414k Google QuickDraw sketches.
• MGT demonstrated faster inference speeds than CNN competitors.
• The proposed approach significantly outperformed all RNN-based models.

Conclusions:

• Representing sketches as graphs and applying GNNs is a novel and effective approach for sketch recognition.
• The multigraph transformer (MGT) offers a promising direction for improving sketch representation learning.