Experiments as Code and its application to VR studies in human-building interaction.

Area of Science:

• Computer Science
• Human-Building Interactions
• Behavioral Science

Background:

• Reproducibility and auditability are critical challenges in scientific experiments, particularly in Human-Building Interactions (HBI).
• Current experimental documentation and management practices often lead to difficulties in reproducing studies and reusing existing work, contributing to the reproducibility crisis.
• Diverse teams and extensive resources required for experiments exacerbate these documentation and reproducibility issues.

Purpose of the Study:

• To introduce and define the Experiments as Code (ExaC) paradigm to address the reproducibility and auditability crisis in scientific research.
• To provide a framework and taxonomy for implementing ExaC, enabling automated provisioning, deployment, management, and analysis of experiments.
• To demonstrate the practical benefits of ExaC through a proof-of-concept in a Human-Building Interactions desktop VR experiment.

Main Methods:

• Defined the Experiments as Code (ExaC) concept and its core principles.
• Developed a taxonomy for practical ExaC implementation components.
• Created a proof-of-concept ExaC implementation for a Human-Building Interactions desktop VR experiment.

Main Results:

• The ExaC paradigm provides automation code for experiment lifecycle management, enhancing reproducibility, auditability, debuggability, reusability, and scalability.
• The proof-of-concept demonstrated the practical advantages of representing experiments 'as code' in an HBI context.
• ExaC facilitates the reuse of experimental components and best practices, mitigating the need to reinvent solutions.

Conclusions:

• Experiments as Code (ExaC) offers a robust solution to the reproducibility and auditability challenges in scientific experimentation.
• The ExaC paradigm promotes efficient research practices by enabling automated, reusable, and scalable experimental workflows.
• Implementing ExaC is crucial for advancing theoretical understanding in fields like Human-Building Interactions and overcoming the reproducibility crisis.