Multistage Spatio-Temporal Networks for Robust Sketch Recognition.

Area of Science:

• Computer Vision
• Machine Learning
• Artificial Intelligence

Background:

• Sketch recognition utilizes spatial and temporal information.
• Current methods often lose details due to late fusion of features.
• Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs) are commonly used but have limitations.

Purpose of the Study:

• To propose a novel framework for sketch recognition with multi-stage spatial and temporal feature interactions.
• To overcome the limitations of existing methods in preserving informative details.
• To improve the robustness and efficiency of sketch recognition systems.

Main Methods:

• A dual-branch network processing stroke arrays and temporal-enriched images (TEI).
• Early-stage enhancement of stroke features using TEI features.
• A final spatio-temporal enhancement module for joint feature refinement.
• A bidirectional temporal-compatible unit to handle abrupt strokes.

Main Results:

• The proposed framework demonstrates improved performance on sketch recognition tasks.
• Experiments on QuickDraw and TU-Berlin datasets validate the method's effectiveness.
• The multi-stage interaction and refinement approach preserves crucial sketch details.

Conclusions:

• The novel framework offers a robust and efficient solution for sketch recognition.
• Integrating spatial and temporal features through multi-stage refinement is key.
• The method effectively addresses challenges posed by stroke order and spatial structures.