MCV-Intention: A Multimodalities and Cross-View Dataset for Human Assembly Intention Recognition.

Area of Science:

• Robotics
• Human-Robot Interaction
• Intelligent Manufacturing

Background:

• Industry 5.0 prioritizes human-centric intelligent manufacturing and human-robot collaboration for mass customization.
• Efficient and safe human-robot collaborative assembly necessitates robots perceiving human operational states.
• Existing vision-based methods for assembly intention recognition are limited by data modalities, real-world process representation, and annotation inconsistencies.

Purpose of the Study:

• Introduce the MCV-Intention dataset, a novel multimodal, cross-view dataset for assembly scene understanding.
• Address limitations in current datasets for human-robot collaborative assembly intention recognition.
• Provide a foundational resource for advancing research in human-robot interaction for Industry 5.0.

Main Methods:

• Collected data from 15 subjects, capturing six modalities and two views per assembly sequence, before and after operator training.
• Developed a comprehensive annotation protocol specifically for assembly intention recognition.
• Detailed the dataset collection process, including hardware, software, and assembly objects used.

Main Results:

• The MCV-Intention dataset comprises multimodal, cross-view data from 15 subjects, offering diverse assembly scenarios.
• The dataset's structure and distribution were analyzed to provide insights into assembly processes.
• Benchmark experiments using state-of-the-art algorithms were conducted to establish performance baselines.

Conclusions:

• The MCV-Intention dataset offers a valuable resource for developing more robust and accurate human intention recognition systems in collaborative assembly.
• This dataset facilitates research towards more effective human-robot collaboration in Industry 5.0 settings.
• The established benchmarks provide a starting point for future advancements in vision-based assembly understanding.