Multimodal PCSC Sensors for Real-Time Temperature and Force Detection Using LRTNet

Multimodal sensors can collect multiple signals and have great potential in robotics and other technical fields. However, such sensors often encounter challenges of signal crosstalk and insufficient real-time performance, particularly in the detection of pressure and temperature, which significantly affect measurement accuracy. To address this issue, a multimodal PCSC sensor was developed. This sensor reduces signal crosstalk by separating force and temperature signals. It uses the pressure-resistance variation of carbon quantum dots (CQDs) to detect force and the thermochromic properties of spiropyran (SP) to detect temperature. When pressure and temperature act on the sensor simultaneously, the resistance increases with pressure and stabilizes when the pressure becomes constant. The response time is 0.4 s. As the temperature rises, the resistance decreases, and the color becomes deeper. Both resistance and color stabilize within 7.5 s. To improve temperature sensing accuracy, a lightweight ResNet-Transformer network (LRTNet) was proposed. This algorithm combines ResNet's ability to extract features and Transformer's ability to model sequences. It efficiently fuses color and resistance signals for temperature detection. Tests on a robotic manipulator for dual recognition of temperature and force showed that LRTNet achieved a runtime of 152.08 ms and a temperature sensing accuracy of 95%. LRTNet improved overall performance by at least 11% compared to traditional algorithms. The sensor and algorithm improved the performance and reliability of multimodal sensors.