Comparative analysis of deep Q-learning algorithms for object throwing using a robot manipulator

Area of Science:

• Robotics
• Artificial Intelligence
• Machine Learning

Background:

• Artificial intelligence (AI) and deep learning, particularly transformers, excel at complex problem-solving.
• Transformers utilize self-attention mechanisms for effective data dependency analysis.
• Deep Q-learning is a key reinforcement learning technique for sequential decision-making.

Purpose of the Study:

• To compare existing deep Q-learning algorithms.
• To propose and evaluate novel deep Q-learning techniques incorporating self-attention and multi-head attention.
• To assess algorithm performance in dynamic robotic control tasks with uncertainties and constraints.

Main Methods:

• Implemented and compared various deep Q-learning algorithms.
• Developed two new methods: structured self-attention with deep Q-learning and multi-head attention with deep Q-learning.
• Evaluated algorithms in a robotic arm manipulation task (throwing a ball) within a simulated environment, including added constraints like joint locks and obstacles.

Main Results:

• The proposed multi-head attention method improved the robot's ability to identify and utilize critical environmental features.
• Algorithms generated accurate joint configurations and trajectories for precise object throwing to unknown targets.
• Temporal difference algorithms effectively addressed robotic joint constraints, enabling solutions within hardware limitations.

Conclusions:

• Self-attention and multi-head attention enhance deep Q-learning for complex robotic control.
• These AI advancements enable robots to perform intelligent, autonomous interactions in dynamic and uncertain environments.
• The developed techniques offer practical solutions for real-world robotic applications with physical constraints.