Improving Real-Time Hand Gesture Recognition with Semantic Segmentation.

Area of Science:

• Computer Vision
• Human-Computer Interaction
• Deep Learning

Background:

• Hand gesture recognition (HGR) is crucial for human-computer interaction (HCI) across various applications.
• Current accurate HGR methods often rely on computationally expensive optical flow, hindering real-time performance.
• There is a need for efficient HGR techniques that maintain high accuracy without excessive computational overhead.

Purpose of the Study:

• To propose a real-time hand gesture recognition method that bypasses optical flow computation.
• To enhance the accuracy of existing efficient HGR methods using hand segmentation masks.
• To evaluate the proposed method's performance on a diverse dataset of touchless screen gestures.

Main Methods:

• Utilized RGB frames combined with hand segmentation masks for HGR.
• Employed a lightweight semantic segmentation network (FASSD-Net) to generate segmentation masks.
• Integrated FASSD-Net with Temporal Segment Networks (TSN) and Temporal Shift Modules (TSM) for gesture recognition.
• Evaluated the approach on the IPN Hand dataset featuring thirteen distinct gestures.

Main Results:

• The proposed method significantly improved the accuracy of TSN and TSM algorithms.
• Real-time performance was maintained, demonstrating computational efficiency.
• The integration of hand segmentation masks effectively compensated for the absence of optical flow.

Conclusions:

• The proposed real-time HGR method offers a computationally efficient and accurate alternative to optical flow-based approaches.
• Hand segmentation masks are effective in boosting HGR accuracy for real-time applications.
• The approach shows promise for enhancing user interaction in touchless screen environments.