Deep-Learning-Based Character Recognition from Handwriting Motion Data Captured Using IMU and Force Sensors.

Area of Science:

• Engineering
• Computer Science
• Human-Computer Interaction

Background:

• Traditional handwriting digitization relies on costly image-based methods (e.g., optical character recognition) or multi-device setups (stylus and smart pad).
• There is a need for affordable, standalone smart pen solutions for efficient handwriting digitization.

Purpose of the Study:

• To develop a compact, deep-learning-based smart digital pen capable of recognizing alphanumeric characters.
• To achieve handwriting recognition using only inertial data from a force sensor, reducing system complexity and cost.

Main Methods:

• A prototype smart pen was constructed integrating an ink chamber, force sensors, an inertial sensor, and a microcomputer.
• Handwritten data for 36 alphanumeric characters were collected from six volunteers.
• Deep learning models, including Vision Transformer (ViT), Deep Neural Network (DNN), Convolutional Neural Network (CNN), and Long Short-Term Memory (LSTM), were trained on the collected data.

Main Results:

• The Vision Transformer (ViT) model achieved the highest validation accuracy at 99.05%.
• The trained model demonstrated promising real-time performance during validation.
• The developed smart pen successfully recognized 36 alphanumeric characters using inertial force sensor data.

Conclusions:

• The developed deep-learning-based smart pen offers a cost-effective and standalone solution for handwriting digitization.
• The Vision Transformer model shows significant potential for accurate handwriting recognition from inertial data.
• This study provides a foundation for future research to expand character recognition capabilities and subject diversity.