Robust Fastener Detection Based on Force and Vision Algorithms in Robotic (Un)Screwing Applications

Area of Science:

• Robotics and Industrial Automation
• Computer Vision
• Manufacturing Technology

Background:

• Industrial maintenance and manufacturing sectors face challenges in automated fastener handling.
• Accurate detection of fastener position and rotation is crucial for robotic (un)screwing tasks.
• Existing vision systems may struggle with varying fastener conditions and lighting.

Purpose of the Study:

• To develop a novel and robust robotic vision system for detecting fastener center and rotation.
• To enable robots to perform (un)screwing tasks on parallel surfaces accurately.
• To evaluate the system's performance across different fastener conditions and lighting environments.

Main Methods:

• A vision system incorporating an industrial camera with dynamic exposure, a tunable liquid crystal lens (TLCL), and near-infrared (NIR) illumination.
• Development of segmentation neural networks (SNN) and vision algorithms for fastener detection.
• Integration with a collaborative robot featuring a force sensor and a virtual program logic controller (PLC) for control and validation.
• Mathematical analysis based on international standards (ISO) and patent reviews for error tolerance determination.

Main Results:

• The system achieves high accuracy, with 99.86% for vertex detection and 100% for center detection.
• Processing time for the SNN and algorithms is 73.91 ms on an Intel Core I9 CPU.
• Translation tolerances increase with fastener size (0.75 for M6 to 2.50 for M24), while rotation tolerance decreases (5.5° for M6 to 3.5° for M24).
• The methodology effectively handles outliers and distorted masks caused by non-constant illumination.

Conclusions:

• The proposed vision system offers a robust and accurate solution for fastener detection in industrial robotic applications.
• The system demonstrates adaptability to various fastener conditions (flawless, worn, rusty) and lighting.
• This development contributes significantly to advancing industrial robotics and automation in manufacturing and maintenance.