A multimodal dataset for human robot collaborative systems: Experimental data

Human-Robot Interaction (HRI) systems are becoming increasingly integral to collaborative industrial and service environments. However, understanding human performance within such settings, particularly in Programming by Demonstration (PbD) frameworks, remains a challenge due to limited availability of comprehensive datasets. This study presents a multimodal dataset designed to assess human performance in HRI applications. The dataset includes both objective and subjective measures collected using various tools within a PbD framework. The data is collected to assess the overall system performance, with a focus on the effectiveness of PbD methods in improving robot learning process. The performance attributes encompass programming efficiency, cognitive workload, usability, and ergonomic assessment of human posture in physical HRI settings. Using tools such as a robot manipulator (UR10e), a vision -based motion tracking system, eye- tracking glasses, NASA Task Load Index (NASA-TLX) and system usability scale (SUS) questionnaire, we recorded interactions between human and robot comprising-robot trajectories, human motion trajectories, participant eye tracking metrices, and participant subjective responses. The data was collected in an experimental setting involving two kinaesthetic robot teaching tasks, under four feedback conditions that provide performance related insights and actionable instructions to improve participant performance. A total of N=28 participants performed three trials of each task using developed Human-Machine Interface (HMI). This dataset is valuable for advancing human-robot teaching interaction and is applicable for developing adaptive HRI systems. The dataset can serve as a significant resource for researchers and machine learning engineers aiming to develop advance HRI systems, improve PbD algorithms, train HRI based machine learning models and facilitate further studies in safety and ergonomic assessment applications in robot learning and collaboration.