Leveraging Imitation Learning on Pose Regulation Problem of a Robotic Fish

Area of Science:

• Robotics
• Artificial Intelligence
• Control Systems

Background:

• Pose regulation for robotic fish is challenging due to conflicting position and orientation objectives.
• Sparse reward schemes in Markov decision processes (MDPs) hinder efficient learning in deep reinforcement learning (DRL).

Purpose of the Study:

• To develop a novel imitation learning (IL) method for robotic fish pose regulation.
• To address the challenges posed by sparse rewards in DRL for robotic control tasks.
• To create an effective demonstrator for generating diverse training data.

Main Methods:

• Formulating the robotic fish pose regulation as an MDP.
• Employing a sparse reward scheme where rewards are only given upon task completion.
• Proposing an IL method that learns DRL-based policies from demonstrations using inverse reward shaping.
• Designing a demonstrator to generate multiple trajectory demonstrations from a single example.

Main Results:

• Simulation results demonstrate the effectiveness of the proposed demonstrator and the state-of-the-art performance of the IL method.
• Experimental deployment on a physical robotic fish validates the method's effectiveness and robustness in real-world conditions, including external disturbances.
• The approach significantly reduces the time and effort required for sample collection.

Conclusions:

• The proposed IL method effectively overcomes sparse reward challenges in robotic fish pose regulation.
• The developed demonstrator streamlines the data generation process for training robotic control policies.
• This work represents a significant advancement in the field of learning for biomimetic underwater robots.