Encoding flexible gait strategies in stick insects through data-driven inverse reinforcement learning

Area of Science:

• Robotics and Biomechanics
• Computational Neuroscience
• Animal Locomotion

Background:

• Stick insects display impressive adaptive walking across varied terrains.
• Mechanisms behind stick insect gait transitions are not fully understood.
• Reinforcement learning (RL) has been used for insect-like gaits, but reward function design is challenging due to probabilistic and continuous gait transitions.

Purpose of the Study:

• To infer the reward function governing stick insect gait selection using maximum entropy inverse RL.
• To clarify principles driving gait transitions and the role of sensory feedback in gait modulation.
• To provide a biologically interpretable framework for gait modeling and bioinspired robotic design.

Main Methods:

• Utilized maximum entropy inverse reinforcement learning (RL) to infer reward functions.
• Incorporated walking dynamic parameters (velocity, direction, acceleration) and antenna joint movements as state variables.
• Validated inferred policies by assessing their ability to reproduce expert trajectories.

Main Results:

• Successfully inferred reward structures that explain stick insect gait selection.
• Demonstrated that stick insect gaits can be learned from observable locomotion states.
• Examined interspecies variations and noncanonical gait patterns, highlighting locomotion flexibility.

Conclusions:

• Sensory feedback plays a crucial role in modulating stick insect gait.
• The data-driven approach provides a biologically interpretable framework for gait modeling.
• Findings facilitate adaptive control strategies for hexapod robots in bioinspired robotic design.