A Novel GAN-Based Synthesis Method for In-Air Handwritten Words

Area of Science:

• Human-Computer Interaction
• Machine Learning
• Signal Processing

Background:

• Micro-electro-mechanical systems (MEMS) enable in-air handwriting technology using inertial sensors.
• Existing research primarily focuses on character-level recognition, leaving word-level recognition with insufficient data and poor generalization.
• Deep learning models require large datasets, which are costly and time-consuming to collect for in-air handwriting.

Purpose of the Study:

• To propose a two-stage synthesis method for generating in-air handwritten word samples.
• To address the challenges of insufficient data and poor generalization in word-level in-air handwriting recognition.
• To improve the accuracy and robustness of in-air handwriting recognition systems.

Main Methods:

• A two-stage synthesis method combining a splicing module guided by an additional corpus and an adversarial learning-based generating module.
• Designing a network capable of handling arbitrary length word samples and focusing on detailed features.
• Conducting experiments on a public dataset to validate the proposed method.

Main Results:

• The proposed method achieved a word-level recognition accuracy of 62.3% on a public dataset, a 62% improvement over models trained solely on the public dataset.
• The synthesis method effectively generates realistic in-air handwritten word samples, even with limited initial data.
• The recognition model trained on synthetic data learned contextual information, enabling recognition of unseen words.

Conclusions:

• The two-stage synthesis method successfully addresses data scarcity and improves generalization for word-level in-air handwriting recognition.
• The approach demonstrates the potential of synthetic data generation in enhancing deep learning models for specialized tasks.
• Future work includes mathematical modeling of character strokes and applying the method to diverse datasets and tasks.