SHARA-WoZ: A multistakeholder framework to evaluate socially assistive robots thought Wizard of the Oz methods

Socially assistive robots (SARs) have emerged as promising solutions for elderly care, offering companionship, cognitive stimulation, and therapeutic interventions. However, their evaluation presents unique challenges due to the multidisciplinary nature of these systems and difficulty assessing them in real situations. This study presents SHARA-WoZ, a multistakeholder framework for evaluating SARs through Wizard of Oz (WoZ) methods.

The framework implements a modular web-based infrastructure comprising three components: robot simulation interface, central server, and desktop wizard control application, supporting autonomous, semi-autonomous, and full-manual operational modes. Evaluation employed a between-subjects study with healthcare professionals and technical experts ( $n=10$ each) using the user experience questionnaire (UEQ), custom usability assessment of system components, and qualitative analysis of post-session interviews.

The framework achieved above-average ratings across all UEQ dimensions. Healthcare professionals showed consistently higher ratings than technical experts, with largest gaps in Efficiency ( $\Delta$ =0.46) and Dependability ( $\Delta$ =0.47), indicating successful alignment with clinical workflows and therapeutic requirements. Qualitative analysis yielded 55 improvement suggestions across Interface Design, New Functionalities, Technical Performance, Integration Features, and Robot Behavior categories.

The findings demonstrate that modular, web-based WoZ infrastructures effectively bridge the gap between research prototypes and real-world deployment requirements. The study confirms the critical importance of multistakeholder evaluation approaches, where healthcare professionals provide clinical perspectives while technical experts contribute optimization insights, ensuring both technical excellence and clinical utility in assistive robotics development.