A Full-Range Proximity-Tactile Sensor Based on Multimodal Perception Fusion for Minimally Invasive Surgical Robots

Minimally invasive surgical robots have received widespread attention due to its numerous advantages. However, the lack of adequate perception capability remains a significant issue for the robots. In this work, a full-range proximity-tactile sensing module has been developed for safe operation of surgical robots, which performs multimodal fusion perception through ultrasonic sensor for long-range proximity detection, capacitive sensor for close-range proximity sensing, and triboelectric sensor for tactile sensing. In order for a minimum sensor size, the ultrasonic sensor is developed based on MEMS piezoelectric micromachined ultrasonic transducers (pMUTs), and the capacitive sensor and triboelectric sensor adopt common structures, which collaborate to achieve accurate proximity-tactile perception. Additionally, a wireless vibration feedback wristband and digital-twin interface are developed to provide multimodal feedback without interfering with operation. Experimental results demonstrates the safety enhancement for surgical robots by the perception and feedback system. Furthermore, the sensing module is applied in preliminary detection of subcutaneous abnormal tissues and the identification accuracy based on the ultrasound echoes and convolutional neural networks is 91.6%, which can provide an initial diagnostic reference. The full-range proximity-tactile sensor holds significant potential for enhancing the safety and detection capability of surgical robots, and promoting the intelligence of robot-assisted minimally invasive surgery.