The Unity Playback System: A tool for visualizing behavioral data in 3D Unity virtual environments

The use of virtual environments, display technology, and eye tracking in behavioral studies has become more widespread in recent years as a result of drops in hardware cost and improvements in hardware mobility. Some studies of human behavior are conducted in 3D virtual environments created using tools such as the Unity game engine and open-source Unity Experiment Framework. However, the raw data from eye trackers, motion capture systems, and input devices (e.g., computer mouse, game controller, steering wheel/foot pedal system) are often noisy and difficult to validate and interpret without visualization. Such visualizations are best implemented in the context of the original environment, but manual implementation to verify experimental results can significantly stall data analysis. To address these issues, we created a generic experimental playback system for Unity that is capable of replaying the frame-by-frame state of the environment without leaving the engine. This GUI-based tool, which we call the Unity Playback System (UPBS) includes options for creating customized visualizations of the behavior of human subjects from different perspectives. We demonstrate the utility of this tool in the context of a simple virtual block stacking task as well as a more complex visually guided steering task, and we discuss how it can facilitate the analysis of gaze data collected in 3D virtual environments.