Person-in-WiFi 3D: Unified Model for 3D WiFi Perception

WiFi signals, in contrast to cameras, offer privacy protection and occlusion resilience for some practical scenarios such as smart homes, elderly care, and virtual reality. Recent years have seen remarkable progress in the estimation of single-person 2D pose, single-person 3D pose, multi-person 2D pose, and single-person mesh. This paper takes a step forward by introducing Person-in-WiFi 3D, a pioneering WiFi system that accomplishes multi-person 3D human perception. Person-in-WiFi 3D has two main updates. Firstly, compared to previous systems that used only one WiFi transmitter and one WiFi receiver, it has a greater number of WiFi devices, which enhances its capability for capturing 3D spatial reflections from multiple individuals. This allows it to respond more flexibly to complicated 3D perception scenarios, achieving a breakthrough from 2D to 3D. Secondly, it utilizes a DETR-like architecture based on Hungarian matching to achieve the end-to-end estimation. It employs a hierarchical refinement strategy from coarse to fine, effectively utilizing features at the global level, human-instance level, and keypoint level, thereby achieving finer-grained estimation and reconstruction. On the basis of these designs, Person-in-WiFi 3D is storage-efficient, accurate, and fast compared to its predecessor. Wedeployed a proof-of-concept system in 4m × 3.5m areas and collected a multi-person human perception dataset Wiception3D of over 97K frames with seven volunteers, encompassing diverse scenes and multi-person scenarios. Wiception3D includes nearly 98k training samples and nearly 8k test samples, annotated for 3D pose estimation and 3D mesh reconstruction tasks. Person-in-WiFi 3D achieves a keypoint localization error of 93mm in 3D pose estimation and 41mm in mesh reconstruction, comparable to cameras and millimeter-wave radars. The project page is at https://aiotgroup.github.io/Person-in-WiFi-3D.