Text Entry Performance and Situation Awareness of a Joint Optical See-Through Head-Mounted Display and Smartphone...

Area of Science:

• Human-Computer Interaction
• Augmented Reality
• Mobile Computing

Background:

• Optical see-through head-mounted displays (OST HMDs) are key for mobile augmented reality (AR) but lack efficient text entry methods.
• Smartphone text entry, while common, poses attentional risks during mobile use.
• Mobile multi-display systems combining OST HMDs and smartphones offer potential for enhanced performance and awareness.

Purpose of the Study:

• To investigate the joint performance of text entry on smartphones with text output on OST HMDs.
• To evaluate the benefits and challenges of integrated OST HMD-smartphone systems for mobile AR text interaction.
• To determine if current combined systems surpass single-device interaction in efficiency and awareness.

Main Methods:

• Conducted five experiments involving 86 participants.
• Assessed text entry performance in a combined OST HMD-smartphone interaction environment.
• Analyzed data to compare joint system performance against single-device benchmarks.

Main Results:

• Text entry on smartphones paired with OST HMDs did not yield significant performance improvements.
• The attentional demands and complexities of the joint system currently outweigh potential benefits.
• Participants did not demonstrate superior performance or situation awareness in the combined system.

Conclusions:

• Integrated OST HMD-smartphone systems for mobile AR text entry face significant challenges.
• Current technology and interaction designs hinder the realization of promised benefits in these multi-display ecologies.
• Further research is needed to overcome limitations and unlock the potential of combined OST HMD-smartphone AR systems.