DrawnNet: Offline Hand-Drawn Diagram Recognition Based on Keypoint Prediction of Aggregating Geometric...

Area of Science:

• Computer Vision
• Machine Learning
• Pattern Recognition

Background:

• Offline hand-drawn diagram recognition aims to digitize sketches for editing.
• Existing methods struggle with diagram structure understanding and rely on time-consuming processes.
• Convolutional Neural Networks (CNNs) excel in various visual tasks.

Purpose of the Study:

• To propose DrawnNet, a unified CNN-based keypoint detector for recognizing symbols and structure in hand-drawn diagrams.
• To address limitations of existing methods in understanding diagram structure and practicability.
• To enable effective digitalization and reconstruction of hand-drawn diagrams.

Main Methods:

• DrawnNet utilizes a CNN architecture based on CornerNet.
• Incorporates novel keypoint pooling modules for geometric feature extraction from polygonal contours.
• Includes an arrow orientation prediction branch using keypoint detection.

Main Results:

• DrawnNet achieved significant recognition rate improvements: 2.4% on FC-A, 2.3% on FC-B, and 1.7% on FA.
• Outperformed state-of-the-art methods across multiple diagram recognition benchmarks.
• Ablation studies confirmed the effectiveness of the proposed method.

Conclusions:

• DrawnNet offers a robust solution for offline hand-drawn diagram recognition.
• The model effectively combines symbol recognition with structural understanding.
• DrawnNet enhances the practicability of digitizing and reconstructing hand-drawn diagrams.