Deep learning-enabled versatile shape perception for soft robots via single-ended multimode fiber

Area of Science:

• Robotics and Intelligent Systems
• Optical Sensing and Metrology
• Artificial Intelligence and Machine Learning

Background:

• Advancements in soft robotics towards embodied intelligent systems necessitate precise proprioception.
• Existing methods for capturing continuous deformations in soft robots are limited, especially in confined interventional settings.
• A universal solution for versatile shape perception in soft robots is currently lacking.

Purpose of the Study:

• To introduce a novel deep learning-enabled versatile shape perception method for soft robots.
• To address the limitations of current proprioception techniques in diverse and confined robotic interactions.
• To establish a foundational framework for enhanced closed-loop control in soft robotic systems.

Main Methods:

• Utilized a single-ended multimode fiber (MMF) with a minimalist reflective optical architecture.
• Leveraged intrinsic optical integration, eliminating the need for complex demodulation units and distal devices.
• Treated chaotic optical speckle fields as data streams encoding high-dimensional shape information, processed by reconfigurable neural decoders.

Main Results:

• Achieved high accuracy (>99%) in discrete state confirmation for soft grippers.
• Demonstrated continuous shape tracking on bionic dexterous hands with a ~5-fold spatial resolution enhancement.
• Enabled intuitive 3D morphological reconstruction of soft surgical robots with an IoU > 0.93.

Conclusions:

• The developed method provides a versatile framework for overcoming hardware adaptability limitations through computation.
• This optical sensing approach establishes a robust foundation for precise proprioception in soft robots.
• The findings pave the way for advanced closed-loop control strategies in digital twins of soft robots.