Fidelity and plausibility of bimanual interaction in mixed reality

When human actors interact with virtual objects the result is often not convincing to a third party viewer, due to incongruities between the actor and object positions. In this study we aim to quantify the magnitude and impact of the errors that occur in a bimanual interaction, that is when an actor attempts to move a virtual object by holding it between both hands. A three stage framework is presented which firstly captures the magnitude of these interaction errors, then quantifies their effect on the relevant third party audience, and thirdly assesses methods to mitigate the impact of the errors. Findings from this work show that the degree of error was dependent on the size of the virtual object and also on the axis of the hand placement with respect to the axis of the interactive motion. In addition, actor hand placement outside and away from the object surface was found to affect the visual plausibility considerably more than when the actor's hands were within the object boundaries. Finally, a method for automatic adaptation of the object size to match the distance between the actor's hands gave a significant improvement in the viewers' assessment of the scene plausibility.