A Hybrid Millimeter-Wave Radar-Ultrasonic Fusion System for Robust Human Activity Recognition with Attention-Enhanced...

Area of Science:

• Human-Computer Interaction
• Sensor Fusion
• Machine Learning for Behavior Recognition

Background:

• Single-sensor systems for human behavior recognition face limitations in environmental robustness and accuracy.
• Existing methods struggle with varying lighting conditions, occlusions, and electromagnetic interference.
• There is a need for non-contact, privacy-preserving solutions adaptable to diverse environments.

Purpose of the Study:

• To develop and evaluate a novel non-contact human behavior recognition system by fusing millimeter-wave (mmWave) radar and ultrasonic array data.
• To overcome the limitations of single-sensor approaches by leveraging complementary sensor strengths.
• To achieve high accuracy and environmental robustness in recognizing human activities like standing, sitting, walking, and falling.

Main Methods:

• A synchronized data acquisition platform integrated 77 GHz mmWave radar and a 40 kHz ultrasonic array.
• Wavelet transform and Short-Time Fourier Transform (STFT) were employed for radar and ultrasound feature extraction, respectively.
• An Attention-CNN-BiLSTM deep learning architecture was utilized for integrated spatial-temporal feature analysis and salient cue enhancement.

Main Results:

• The fused sensor system achieved a mean class accuracy of 98.6% on 1600 synchronized behavior sequences.
• The system demonstrated strong subject-wise generalization capabilities, outperforming single-sensor baselines and conventional deep learning models.
• The proposed method proved effective across various human behaviors including standing, sitting, walking, and falling.

Conclusions:

• The fusion of mmWave radar and ultrasonic array provides a robust and accurate solution for non-contact human behavior recognition.
• The Attention-CNN-BiLSTM architecture effectively integrates multi-modal data for enhanced performance.
• This privacy-preserving, lighting-agnostic system holds significant potential for applications in smart homes, healthcare, and surveillance.