Comparison of deep learning-based methods in multimodal anomaly detection: A case study in human-robot collaboration

Area of Science:

• Robotics
• Artificial Intelligence
• Machine Learning

Background:

• Human-Robot Collaboration (HRC) leverages combined human and robot strengths.
• Internal modeling errors and external perturbations pose risks in HRC, necessitating anomaly detection.
• Multimodal anomaly detection is crucial for identifying unexpected events in HRC.

Purpose of the Study:

• To evaluate and compare six deep learning-based methods for multimodal anomaly detection in HRC.
• To assess detection accuracy, multi-modality combinations, and anomaly time bias.
• To identify the most effective method for real-world HRC applications.

Main Methods:

• Six representative deep learning methods were trained on non-anomalous multimodal signals (force, torque, velocity, tactile, kinematic).
• Anomaly detection was performed using a predefined threshold.
• Performance was evaluated during a human-robot kitting experiment involving six skills.

Main Results:

• The LSTM-DAGMM based detector achieved higher accuracy and efficiency compared to other evaluated methods.
• Receiver Operating Characteristic (ROC) and other metrics were analyzed across various multi-modality combinations and anomaly biases.
• The study identified optimal settings for multimodal anomaly detection.

Conclusions:

• The LSTM-DAGMM model shows significant promise for robust multimodal anomaly detection in HRC.
• Comparative analysis and verification in specific application cases are vital for advancing HRC safety.
• Further research can optimize multimodal anomaly detection for enhanced HRC performance.