Multisensory visual servoing by a neural network

Area of Science:

• Robotics
• Computer Vision
• Machine Learning

Background:

• Traditional robot motion determination relies on complex sensor calibration (camera, hand-eye), which is computationally intensive and challenging with diverse sensors.
• Existing methods require recalibration for new tasks or sensor configurations, limiting adaptability.

Purpose of the Study:

• To develop a calibration-free neural network approach for robot end-effector motion determination using multi-sensory data.
• To enable direct transformation from sensory inputs to robot motions, simplifying the process and improving efficiency.
• To achieve adaptability to changing goal positions without network retraining.

Main Methods:

• A multilayer feedforward neural network was employed, taking camera images and laser range data as input.
• A recursive motion strategy and network correction were used to relax the need for precise transformation learning.
• Sensor fusion was achieved by integrating data from different sensor modalities into the neural network.

Main Results:

• The proposed neural network approach successfully determined robot end-effector motion without requiring sensor or hand-eye calibration.
• The method demonstrated effective sensor fusion of camera and laser data.
• The system allowed for changes in goal positions without the need for retraining the neural network.

Conclusions:

• The calibration-free neural network approach offers a practical and efficient solution for robot motion determination.
• This method simplifies robot system integration and enhances adaptability to dynamic environments and tasks.
• The findings highlight the potential of neural networks for advanced sensor fusion and control in robotics.