Heterogeneous sensing in a multifunctional soft sensor for human-robot interfaces

Soft sensors have been playing a crucial role in detecting different types of physical stimuli to part or the entire body of a robot, analogous to mechanoreceptors or proprioceptors in biology. Most of the currently available soft sensors with compact form factors can detect only a single deformation mode at a time due to the limitation in combining multiple sensing mechanisms in a limited space. However, realizing multiple modalities in a soft sensor without increasing its original form factor is beneficial, because even a single input stimulus to a robot may induce a combination of multiple modes of deformation. Here, we report a multifunctional soft sensor capable of decoupling combined deformation modes of stretching, bending, and compression, as well as detecting individual deformation modes, in a compact form factor. The key enabling design feature of the proposed sensor is a combination of heterogeneous sensing mechanisms: optical, microfluidic, and piezoresistive sensing. We characterize the performance on both detection and decoupling of deformation modes, by implementing both a simple algorithm of threshold evaluation and a machine learning technique based on an artificial neural network. The proposed soft sensor is able to estimate eight different deformation modes with accuracies higher than 95%. We lastly demonstrate the potential of the proposed sensor as a method of human-robot interfaces with several application examples highlighting its multifunctionality.