Skill Information Representation Imitation Learning for Long-Horizon Dexterous Robot Micromanipulation of Deformable...

Robots performing collaborative long-horizon dexterity cell micromanipulation tasks are challenging and practically significant, such as peeling cell membranes, which is considered one of the most technically demanding procedures. The imitation learning (IL) approach is expected to address the challenges of multitask coupling and object modeling difficulties in long-horizon tasks. Existing IL algorithms suffer from compounding error as they perform only a simple mapping of the task environment space to the action space. In this article, we propose a skill information representation IL (SIRIL) algorithm for long-horizon dexterous robot micromanipulation tasks. First, SIRIL extracts the discrete latent codes of the expert video frames by the VQ-GAN encoder, and the distribution of the latent codes is modeled by an autoregressive transformer. SIRIL quantifies the representation of the expert's skill information by computing the log-likelihood of the latent discrete codes, which allows for the extraction of safe action constraints. Finally, SIRIL predicts actions by integrating actions from previous time steps, and actions that satisfy the safety constraints are executed, which effectively suppresses compounding error. Real experiments show that the SIRIL algorithm can complete the deformable zebrafish embryonic cells dexterous membrane stripping surgery. Ablation studies further confirmed SIRIL's high efficiency in various subtasks, including PushCell, GraspCell, and PeelCell, which achieves an average accuracy of 86.7% and a high final success rate of 64.7%, significantly outperforming existing algorithms. Code is available at https://github.com/zycrobot/SIRIL.