ARCHIE++ : A Cloud-Enabled Framework for Conducting AR System Testing in the Wild.

Area of Science:

• Computer Science
• Human-Computer Interaction
• Software Engineering

Background:

• Current augmented reality (AR) system testing often lacks real-world context.
• Collecting in-situ usability feedback alongside performance data is challenging.
• Existing methods struggle with scalability and reproducibility in wild testing.

Purpose of the Study:

• To introduce ARCHIE++, a novel framework for AR system testing in real-world environments.
• To address the challenges of collecting and integrating user feedback with system performance data.
• To improve the identification of edge cases and enhance the robustness of AR systems.

Main Methods:

• Reviewed current trends in human testing of AR systems from leading conferences.
• Identified key challenges in adopting AR testing practices for in-the-wild scenarios.
• Designed ARCHIE++ as a cloud-enabled, device-agnostic framework for data aggregation.

Main Results:

• ARCHIE++ aggregates in-situ usability feedback with concurrent system performance data.
• The framework facilitates the identification of edge cases during unconstrained AR usage.
• Case studies demonstrate ARCHIE++'s utility across various AR testing scenarios with limited overhead.

Conclusions:

• ARCHIE++ offers a scalable and reproducible solution for AR system testing in the wild.
• The framework enhances developer understanding of environmental conditions impacting AR performance.
• ARCHIE++ provides valuable insights for improving AR system design and user experience.