Person-in-WiFi 3D: Unified Model for 3D WiFi Perception

Area of Science:

• Computer Vision
• Wireless Sensing
• Human-Computer Interaction

Background:

• WiFi sensing offers privacy and occlusion resilience, advancing applications in smart homes and elderly care.
• Previous WiFi systems primarily focused on single-person 2D or 3D pose estimation.
• Significant progress has been made in human pose and mesh estimation using various sensing modalities.

Purpose of the Study:

• To introduce Person-in-WiFi 3D, a novel WiFi system for multi-person 3D human perception.
• To overcome limitations of single-transmitter/receiver WiFi systems for complex 3D scenarios.
• To enable accurate and efficient 3D pose and mesh reconstruction of multiple individuals using WiFi signals.

Main Methods:

• Employed a multi-device WiFi system for enhanced 3D spatial reflection capture.
• Utilized a DETR-like architecture with Hungarian matching for end-to-end estimation.
• Implemented a hierarchical refinement strategy (global, instance, keypoint levels) for fine-grained reconstruction.

Main Results:

• Achieved a keypoint localization error of 93mm for 3D pose estimation.
• Attained a mesh reconstruction error of 41mm.
• Developed and validated the system on the Wiception3D dataset, comprising over 97K frames.

Conclusions:

• Person-in-WiFi 3D demonstrates storage efficiency, accuracy, and speed compared to prior systems.
• The system achieves performance comparable to camera and millimeter-wave radar systems.
• This work represents a significant advancement in multi-person 3D human perception using WiFi sensing.