Millimeter wave gesture recognition using multi-feature fusion models in complex scenes

Zhanjun Hao ^1,2, Zhizhou Sun ³, Fenfang Li ⁴

⁰College of Computer Science and Engineering, Northwest Normal University, Lanzhou, 730070, China. zhanjunhao@126.com.

Scientific Reports +

|

June 14, 2024

View abstract on PubMed

Summary

This study introduces a novel millimeter wave radar-based gesture recognition system for complex environments. The multi-feature fusion approach and lightweight neural network achieve high accuracy, overcoming limitations of existing methods.

Area Of Science

Human-Computer Interaction
Signal Processing
Machine Learning

Background

Gesture recognition is crucial for smart homes and human-computer interaction.
Existing methods face limitations in user experience, visual environments, and recognition detail.
Millimeter wave radar offers high precision and bandwidth for gesture recognition.

Purpose Of The Study

To propose a robust gesture recognition method for complex scenes using millimeter wave radar.
To address limitations of current gesture recognition techniques.
To enhance gesture recognition accuracy and reliability.

Main Methods

Collected diverse data and filtered clutter to improve signal-to-noise ratio (SNR).
Extracted multi-features: range-time map (RTM), Doppler-time map (DTM), and angle-time map (ATM).
Developed a multi-CNN-LSTM neural network for fused feature recognition.

Main Results

The multi-feature fusion enhanced feature richness and expression.
The lightweight multi-CNN-LSTM model demonstrated effectiveness.
Achieved 97.28% recognition accuracy for 14 gestures in complex scenarios.

Conclusions

The proposed method exhibits generalization, adaptability, and robustness in complex environments.
Multi-feature fusion with millimeter wave radar is effective for advanced gesture recognition.
The system overcomes previous limitations, offering practical applications.