Intuition-guided Reinforcement Learning for Soft Tissue Manipulation with Unknown Constraints

Intraoperative soft tissue manipulation is a critical challenge in autonomous robotic surgery. Furthermore, the intricate in vivo environment surrounding the target soft tissues poses additional hindrances to autonomous robotic decision-making. Previous studies assumed the grasping point was known and the target deformation could be achieved. The constraints were assumed to be constant during the operation, and there were no obstacles around the soft tissue. To address these problems, an intuition-guided deep reinforcement learning framework based on soft actor-critic (ID-SAC) was proposed for soft tissue manipulation under unknown constraints. The SAC algorithm is automatically activated upon encountering an obstacle, and the designed intuitive manipulation (IM) strategy is used to pull soft tissues toward the target shape directly when the obstacle is distant. A regulator factor is designed as an action within this framework to coordinate the IM approach and the SAC network. A reward function is designed to balance the exploration and exploitation of large deformations. Simultaneously, we proposed an autonomous grasp point selection neural network to prevent the impractical selection of grasp points, ensuring they can reach the target while avoiding grasping lesions and constrained areas. Successful simulation results confirmed that the proposed framework can manipulate the soft tissue while avoiding obstacles and adding new positional constraints. Compared with the SAC algorithm, the proposed framework can markedly increase the robotic soft tissue manipulation ability by automatically adjusting the regulator factors.