Behavioral and Physiological Responses to Sensory and Cognitive Demands in an Immersive Virtual Reality Shopping Task...

Immersive virtual reality (IVR) is increasingly used for task training. Adjusting sensory and cognitive complexity to individual capacity is important for effective training, yet it remains unclear how varying visual, auditory, and cognitive demands influence behavioral responses, physiological activity, and task performance during goal-directed activities in IVR.

This study aimed to examine how 2 levels (low and high) of visual, auditory, and mental demands affect self-reported mental demand and effort (MDE), task performance, heart rate, gaze behavior, and head movements during a virtual grocery shopping task.

A within-subjects study was conducted with 22 unimpaired university students (11 women, 11 men; aged 23-27 years) recruited via convenience sampling at TU Delft, the Netherlands. Participants completed a grocery shopping task (collecting products from shelves during 122-second trials) under 7 conditions: baseline, visual low/high (background characters), auditory low/high (background sounds), and mental low/high (0-back/2-back tasks), in random order. Outcome measures included MDE (0-100), task performance (products placed in the cart), mean heart rate (from electrocardiography), head-stillness duration, and gaze duration on areas of interest (via integrated eye-tracking). Two-tailed paired-samples t tests with Bonferroni correction (α=.008) were supplemented by linear mixed-effects models.

The mental high condition produced the largest effects: MDE increased relative to baseline (mean difference [M_diff] 52.73, 95% CI 44.01-61.44; t₂₁=12.58; P<.001), performance decreased (M_diff -8.00 products, 95% CI -9.57 to -6.43; t₂₁=-10.58; P<.001), head-stillness duration increased (M_diff 13.54 s, 95% CI 9.94-17.14; t₂₁=7.83; P<.001), and gaze on the shopping list increased (M_diff 9.89 s, 95% CI 5.36-14.43; t₂₁=4.54; P<.001). Auditory high demands increased MDE (M_diff 17.05, 95% CI 9.15-24.94; t₂₁=4.49; P<.001) without significantly affecting performance. Visual high demands redistributed gaze away from the main shelves (M_diff -6.11 s, 95% CI -8.70 to -3.52; t₂₁=-4.91; P<.001) without reducing performance. Heart rate increased under mental low demand (M_diff 4.02 bpm, 95% CI 1.69-6.34; t₂₁=3.59; P=.002); the mental high effect did not reach corrected significance in the paired-samples test (P=.02), possibly due to reduced physical activity in this condition. Under high cognitive load, participants spent more time checking the shopping list and showed greater head-stillness, consistent with task-serialization and memory-offloading strategies.

Cognitive demands strongly impacted workload, performance, and behavior, while sensory demands affected attention distribution and perceived workload, with no significant performance decrements at the Bonferroni-corrected threshold. By comparing 2 demand levels against a no-demand baseline and incorporating gaze and head-movement measures not previously examined, this study extends prior work in this domain. The findings suggest IVR environments can incorporate sensory richness without large performance reductions in unimpaired young adults, and that behavioral strategies under cognitive load may support real-world workload monitoring and adaptive task adjustment, pending clinical validation.