Hybrid User Interfaces: Past, Present, and Future of Complementary Cross-Device Interaction in Mixed Reality

Area of Science:

• Human-Computer Interaction
• Computer Science

Background:

• Hybrid User Interfaces (HUIs) integrate diverse devices for complementary functions.
• Cross-device interaction between 2D displays and mixed reality (MR) environments is a key HUI area.
• Existing research on MR-based HUIs lacks a unified understanding and consistent terminology.

Purpose of the Study:

• To establish a cohesive understanding and consistent terminology for hybrid user interfaces.
• To systematically survey and categorize HUIs combining 2D displays and mixed reality.
• To analyze challenges, definitions, and future opportunities in HUI research.

Main Methods:

• Conducted a systematic survey of hybrid user interface research.
• Developed a taxonomy for HUIs integrating conventional displays and mixed reality environments.
• Analyzed historical and current research trends, definitions, and challenges.

Main Results:

• A taxonomy categorizing HUIs that combine 2D displays and mixed reality environments.
• Identification of unique design possibilities and challenges in these specific HUIs.
• A review of the evolution of HUI definitions and terminology over 30 years.

Conclusions:

• A unified understanding of hybrid user interfaces, particularly those involving mixed reality, is needed.
• The proposed taxonomy provides a framework for future HUI research and development.
• Addressing identified challenges and opportunities will advance the field of hybrid user interfaces.