Beyond Signatures: Leveraging Sensor Fusion for Contextual Handwriting Recognition.

Area of Science:

• Biometrics
• Human-Computer Interaction
• Machine Learning

Background:

• Current handwriting biometrics often focus solely on signatures and specific sensors.
• Existing methods are predominantly verification-based, not recognition-based.
• There's a need for broader handwriting analysis and sensor-agnostic approaches.

Purpose of the Study:

• To develop a novel handwriting-based user recognition system.
• To incorporate sensor fusion for enhanced biometric data collection.
• To analyze various handwriting forms beyond signatures and different input modalities.

Main Methods:

• Utilized sensor fusion with integrated and external sensors for dynamic writing data.
• Collected a dataset from 60 participants using styluses and fingers on touchscreens.
• Employed custom Convolutional Neural Network (CNN) architectures for feature extraction and classification.

Main Results:

• Achieved high accuracies: 0.982 for signatures, 0.927 for words, 0.884 for short sentences, and 0.661 for individual letters.
• Investigated the impact of input modality (stylus vs. finger) and training data size on accuracy.
• Conducted an ablation study to assess the contribution of individual sensors in the fusion setup.

Conclusions:

• The proposed sensor fusion approach significantly improves handwriting biometrics.
• Analyzing diverse handwriting forms (words, sentences) alongside signatures enhances recognition capabilities.
• The system demonstrates robust performance across different input modalities and data sizes.