Network Layer Analysis for a RL-Based Robotic Reaching Task.

Area of Science:

• Robotics
• Machine Learning
• Neural Networks

Background:

• Pretraining end-to-end reinforcement learning (RL) networks on general tasks accelerates training for specific robotic applications.
• The reusability of feature extractors and hierarchical organization within these RL networks, analogous to convolutional neural networks, remains an open question.

Purpose of the Study:

• To analyze intrinsic neuron activation in RL networks trained for robot manipulator target reaching with varying joint numbers.
• To investigate the potential for reusable feature extractors and hierarchical organization in RL networks for robotics.

Main Methods:

• Analysis of individual neuron activation distribution within networks trained for robot manipulator target reaching.
• Introduction of a pruning algorithm to enhance network information density and identify neuron activation pattern correlations.
• Probing projections of neuron activation across networks trained for robot kinematics of differing complexity.

Main Results:

• Input and output network layers exhibit more distinct neuron activation compared to inner layers.
• The developed pruning algorithm significantly reduces network size and increases neuron activation distance while maintaining high performance.
• Networks trained on robot kinematics with minor joint number differences show higher layer-wise projection accuracy.

Conclusions:

• Reinforcement learning networks demonstrate characteristics of reusable feature extractors, particularly evident in layer-wise projection accuracy.
• Robot kinematics with greater complexity lead to dominant projections in the initial network layers, suggesting a hierarchical processing of kinematic information.
• The proposed pruning method effectively optimizes RL networks for robotic applications by enhancing information density and reducing computational load.