KD-Net: Continuous-Keystroke-Dynamics-Based Human Identification from RGB-D Image Sequences.

Area of Science:

• Computer Vision
• Biometrics
• Human-Computer Interaction

Background:

• Keystroke dynamics traditionally analyzes key press/release events for user identification.
• Existing methods often overlook rich visual cues from hand movements during typing.

Purpose of the Study:

• To explore a novel visual modality of keystroke dynamics for human identification using RGB-D sensors.
• To create and validate a new dataset (KD-MultiModal) for this purpose.
• To evaluate deep learning models for person identification using RGB-D data.

Main Methods:

• Developed methods for extracting Regions of Interest (RoIs) from hand and keyboard areas in RGB-D data.
• Utilized deep neural networks (RGB-KD-Net, D-KD-Net, RGBD-KD-Net) to learn features from RGB, depth, and combined data.
• Investigated human identification performance using point clouds derived from depth images.

Main Results:

• The proposed method using RGB-D images achieved the highest accuracy of 99.44% on unseen real-world data.
• The KD-MultiModal dataset, containing 243.2K frames, captures hand shape, posture, and typing dynamics.
• Deep learning models effectively learned distinguishing features from the visual keystroke dynamics data.

Conclusions:

• Visual keystroke dynamics using RGB-D sensors is a viable and highly accurate method for human identification.
• The KD-MultiModal dataset provides a valuable resource for advancing research in this area.
• The study demonstrates the superiority of combined RGB-D data for robust person identification.