DUAL-Net: Joint Domain-Invariant and User-Adaptive Feature Learning for Gesture Recognition

Area of Science:

• Computer Science
• Human-Computer Interaction
• Ubiquitous Computing

Background:

• WiFi-based gesture recognition offers contactless sensing but suffers from performance degradation due to environmental and user variations.
• Existing methods often overlook crucial user-specific characteristics in favor of domain-invariant representations, limiting cross-user accuracy.

Purpose of the Study:

• To develop a robust WiFi-based gesture recognition system that overcomes cross-user performance degradation.
• To propose a novel dual-branch framework, DUAL-Net, that jointly models domain-invariant and user-adaptive features.

Main Methods:

• DUAL-Net utilizes a contrastive fusion learning (CFL) module with modality-specific encoders for complementary WiFi and vision representations.
• A spatial matrix difference (SMD)-guided cross-modal generation (CMG) module generates user-adaptive WiFi features using skeletal structural priors.
• A two-stage learning framework enables offline adaptation to reduce online computational overhead.

Main Results:

• DUAL-Net demonstrates superior cross-user gesture recognition performance compared to existing single-modality and multimodal methods.
• SMD-guided conditioning improved recognition accuracy by up to 8.79% compared to diffusion generation without structural guidance.
• Experiments were validated on the MM-Fi dataset and a self-collected dataset.

Conclusions:

• DUAL-Net effectively addresses the challenge of user-specific differences in WiFi-based gesture recognition.
• The proposed framework achieves state-of-the-art performance in cross-user scenarios, enhancing the practicality of ubiquitous computing applications.
• Integrating structural priors via SMD-guided CMG significantly boosts recognition accuracy.