Robust Multimodal Indirect Sensing for Soft Robots Via Neural Network-Aided Filter-Based Estimation

Sensory data are critical for soft robot perception. However, integrating sensors to soft robots remains challenging due to their inherent softness. An alternative approach is indirect sensing through an estimation scheme, which uses robot dynamics and available measurements to estimate variables that would have been measured by sensors. Nevertheless, developing an adequately effective estimation scheme for soft robots is not straightforward. First, it requires a mathematical model; modeling of soft robots is analytically demanding due to their complex dynamics. Second, it should perform multimodal sensing for both internal and external variables, with minimal sensors, and finally, it must be robust against sensor faults. In this article, we propose a recurrent neural network-based adaptive unscented Kalman filter (RNN-AUKF) architecture to estimate the proprioceptive state and exteroceptive unknown input of a pneumatic-based soft finger. To address the challenge in modeling soft robots, we adopt a data-driven approach using RNNs. Then, we interconnect the AUKF with an unknown input estimator to perform multimodal sensing using a single embedded flex sensor. We also prove mathematically that the estimation error is bounded with respect to sensor degradation (noise and drift). Experimental results show that the RNN-AUKF achieves a better overall performance in terms of accuracy and robustness against the benchmark method. The proposed scheme is also extended to a multifinger soft gripper and is robust against out-of-distribution sensor dynamics. The outcomes of this research have immense potentials in realizing a robust multimodal indirect sensing in soft robots.