Grasping Real and Virtual Objects in AR: Study of Carry-Over Effects on Grasping Anticipatory Behavior

When alternatively interacting with virtual and real objects in an Augmented Reality (AR), users experience sensory discrepancies, notably differences in visual and tactile feedback. While prior work has demonstrated that interacting with both virtual and real objects leads to a decrease in grasping performance for real objects, the underlying impact on the grasping process itself remains unclear. To explore how virtual object interactions can influence subsequent interactions with real objects, we conducted a pick-and-place experiment (N = 28), where participants alternated interactions between real and virtual cylinders using two common virtual grasping methods, a pinch gesture or a physics-based grasping method. We measured the grasp anticipatory process, i.e. pre-shaping, on subsequent interactions with real objects (maximum grip aperture and time to peak aperture) and transition time. Results show that virtual interactions, no matter the virtual grasping method used, influenced pre-shaping towards subsequent real-object reach motion. Moreover, the physics-based grasping method significantly showed the largest grip aperture and the earliest preparation for subsequent interactions with real objects, suggesting greater grasp uncertainty. Taken together our experimental results demonstrate the importance of investigating virtual-real interactions in AR setups, highlighting the relevance of using the pre-shaping analysis as a formative metric for quantifying the modulation of the users' grasping process during virtual-to-real transitions.