Deep Learning Framework for Controlling Work Sequence in Collaborative Human-Robot Assembly Processes.

Area of Science:

• Robotics
• Artificial Intelligence
• Computer Vision

Background:

• Existing human-robot collaboration (HRC) solutions are inefficient, relying on human state or gestures, which increases task time.
• This limitation hinders the pace of human labor and reduces the attractiveness of current HRC systems.

Purpose of the Study:

• To introduce a new HRC framework for managing assembly processes executed by humans and robots simultaneously or individually.
• To leverage deep learning models using only RGB camera data for workspace and human action prediction to manage assembly.

Main Methods:

• Developed an HRC framework utilizing deep learning models, specifically Convolutional Neural Network (CNN) architectures.
• Implemented and compared four CNN models: Faster R-CNN ResNet-50, ResNet-101, YOLOv2, and YOLOv3.
• Validated the framework using an industrial HRC demonstrator for mechanical component assembly.

Main Results:

• The HRC framework based on the YOLOv3 structure demonstrated the best performance.
• Achieved a mean average precision of 72.26% with the YOLOv3 framework.
• The HRC industrial demonstrator successfully completed all assembly tasks within the specified time window.

Conclusions:

• The proposed HRC framework is effective for industrial assembly applications.
• The deep learning approach using RGB data offers a promising direction for improving HRC efficiency.
• The YOLOv3-based framework provides a robust solution for real-time collaborative assembly management.