Intelligent Gesture Recognition Based on Screen Reflectance Multi-Band Spectral Features.

Area of Science:

• Human-Computer Interaction (HCI)
• Computer Vision
• Signal Processing

Background:

• Digitalization trends emphasize intuitive human-computer interaction (HCI) methods.
• Gesture recognition via screens is a key area for advancing HCI.
• Existing methods may face challenges with varying environmental lighting.

Purpose of the Study:

• To propose a novel gesture recognition method combining spectral and spatial features.
• To develop an RGB three-channel spectral gesture recognition system.
• To enhance the accuracy and stability of gesture recognition for HCI.

Main Methods:

• Utilizing multi-band spectral features and spatial characteristics of screen-reflected light.
• Developing a system with a display screen and narrowband spectral receivers.
• Formulating an RGB multi-channel convolutional neural network long short-term memory (CNN-LSTM) model.

Main Results:

• Achieved 99.93% accuracy in darkness and 99.89% in illuminated conditions.
• Demonstrated stable and accurate gesture recognition across different lighting environments.
• Integrated multidimensional features from frequency and spatial domains for enhanced classification.

Conclusions:

• The proposed spectral gesture recognition method offers high accuracy and stability.
• The developed RGB CNN-LSTM model is effective for gesture-based HCI.
• This intelligent method has broad applications for interactive screens like computers and mobile devices.