Sketch-R2CNN: An RNN-Rasterization-CNN Architecture for Vector Sketch Recognition

Area of Science:

• Computer Vision
• Machine Learning
• Artificial Intelligence

Background:

• Sketches are often stored as vector data, but current recognition methods typically rasterize them into images.
• This rasterization process can lead to information loss and suboptimal recognition performance.

Purpose of the Study:

• To propose a novel end-to-end network architecture, Sketch-R2CNN, that fully utilizes the vector format of sketches for recognition.
• To improve the accuracy and efficiency of sketch recognition by directly processing vector data.

Main Methods:

• Developed Sketch-R2CNN, an architecture combining a Recurrent Neural Network (RNN) for per-point feature extraction in vector space.
• Introduced a differentiable neural line rasterization module to convert vector sketches and features into multi-channel feature maps.
• Utilized a Convolutional Neural Network (CNN) to extract convolutional features from the generated feature maps in pixel space.

Main Results:

• The RNN-Rasterization design consistently improved performance compared to CNN baselines.
• Sketch-R2CNN demonstrated substantial outperformance over existing state-of-the-art sketch recognition methods.
• The end-to-end learning approach enabled effective feature extraction directly from vector data.

Conclusions:

• Sketch-R2CNN offers a significant advancement in sketch recognition by directly leveraging vector data.
• The proposed neural rasterization module is effective for converting vector information into a format suitable for CNNs.
• This approach paves the way for more accurate and robust sketch recognition systems.