A comparison of natural user interface and graphical user interface for narrative in HMD-based augmented reality

Area of Science:

• Human-Computer Interaction
• Immersive Storytelling
• Augmented Reality

Background:

• Storytelling mediums have evolved to include Virtual Reality (VR) and Augmented Reality (AR).
• Head-Mounted Display (HMD) AR technology, like Microsoft HoloLens, offers new narrative possibilities.
• User experience with narration in HMD-based AR remains under-explored.

Purpose of the Study:

• To investigate interactive narrative within HMD-based AR.
• To compare the influence of Natural User Interface (NUI) and Graphical User Interface (GUI) on user presence, narrative engagement, and reflection.
• To develop design guidelines for AR narrative applications.

Main Methods:

• Developed an AR narrative application, "The AR Journey," using Microsoft HoloLens.
• Employed a between-subject experimental design comparing NUI and GUI interaction methods.
• Conducted qualitative interviews to gather user feedback on the narrative experience.

Main Results:

• NUI showed significant advantages in creating user presence for individuals without 3D Role Playing Game (RPG) experience.
• GUI was superior in enhancing presence and narrative engagement for users with 3D RPG experience.
• Qualitative data indicated an acceptable overall narrative experience, but hardware limitations and virtuality-reality mismatches impacted user experience.

Conclusions:

• Interface design significantly influences user presence and engagement in HMD-based AR narratives.
• Tailoring interface design to user experience (e.g., gaming background) is crucial for optimizing AR storytelling.
• Addressing hardware limitations and sensory mismatches is essential for improving future HMD-based AR narrative applications.