Translating human information into robot tasks: action sequence recognition and robot control based on human motions.

Area of Science:

• Robotics and Human-Computer Interaction
• Biomedical Engineering
• Artificial Intelligence

Background:

• Wearable cyborgs require reliable power sources, driving battery innovation.
• Current research relies on manual, trial-and-error processes, demanding significant researcher time and effort.
• Reproducibility in research is challenged by the manual nature of experimental procedures.

Purpose of the Study:

• To develop a robot system capable of performing sequential tasks traditionally done by researchers.
• To reduce researcher workload and enhance the reproducibility of trial-and-error research.
• To enable robots to learn and execute complex procedures from human demonstrations.

Main Methods:

• A non-contact system was developed to capture 3D skeletal motion data over time.
• An action sequence recognition model was created using skeletal data and object detection, independent of background.
• Human motion and task information were translated for robot execution of sequential tasks.

Main Results:

• The system achieved high accuracy in recognizing human-performed tasks (95.39% Edit score, 0.951 F1@10 score).
• The robot successfully adapted to changes in work processes in 50% of trials.
• The robot seamlessly executed sequential tasks after learning from human demonstrations.

Conclusions:

• The proposed system demonstrates the feasibility of robots learning and performing complex research tasks.
• This technology can significantly reduce manual labor in research settings.
• The system enhances reproducibility and efficiency in scientific experimentation.