Too Close for Comfort: Continuous Anxiety Decoding from Physiology and Embodied Signals in Social VR

Immersive social VR enables fine-grained manipulation of anxiety-inducing encounters-crowds, proxemic intrusions, and confrontational agents-yet how physiological, behavioral, and environmental signals co-vary with and could decode moment-to moment anxiety in these settings remains largely unexplored. To address this, here we present a study of N = 108 participants navigating five self-paced, multi-agent social VR scenes, under synchronized physiological (EDA, PPG, RSP, eye-tracking), locomotion (speed, head rotation), and proxemics (agent distance, count) recording, with continuous anxiety rated retrospectively. Four converging validity analyses validate this continuous retrospective self-report as a sensitive measure of state social anxiety, with consistent physiological co-variation confirming genuine affective responses. Importantly, we systematically evaluate three settings for anxiety prediction: within-person decoding, cross-participant generalization, and causal prospective prediction with a ground-truth autoregressive prior (the participant's previous-window anxiety). Within individuals, deep sequence modeling (BiLSTM) achieves excellent prediction performance. In the cross-participant setting, however, performance drops substantially. For the first two decoding tasks, combining all modalities yields the best overall performance. In the most challenging prospective prediction setting, predictability is dominated by the autoregressive prior, with sensors providing only a modest incremental gain in the across-person classical setting; we chart how this varies across calibration length and classifier. Our results establish a quantitative baseline for continuous anxiety decoding in ecological social VR and highlight cross-participant transfer as a key open challenge.