Tactile sensors, which provide information about the physical properties of objects, are an essential component of robotic systems. The visuotactile sensing technology with the merits of high resolution and low cost has facilitated the development of robotics from environment exploration to dexterous operation. Over the years, several reviews on visuotactile sensors for robots have been presented, but few of them discussed the significance of signal processing methods to visuotactile sensors. Apart from ingenious hardware design, the full potential of the sensory system toward designated tasks can only be released with the appropriate signal processing methods. Therefore, this paper provides a comprehensive review of visuotactile sensors from the perspective of signal processing methods and outlooks possible future research directions for visuotactile sensors.

Advancement in human-robot interaction (HRI) is essential for the development of intelligent robots, but there lack paradigms to integrate remote control and tactile sensing for an ideal HRI. In this study, inspired by the platypus beak sense, we propose a bionic electro-mechanosensory finger (EM-Finger) synergizing triboelectric and visuotactile sensing for remote control and tactile perception. A triboelectric sensor array made of a patterned liquid-metal-polymer conductive (LMPC) layer encodes both touchless and tactile interactions with external objects into voltage signals in the air, and responds to electrical stimuli underwater for amphibious wireless communication. Besides, a three-dimensional finger-shaped visuotactile sensing system with the same LMPC layer as a reflector measures contact-induced deformation through marker detection and tracking methods. A bioinspired bimodal deep learning algorithm implements data fusion of triboelectric and visuotactile signals and achieves the classification of 18 common material types under varying contact forces with an ac- curacy of 94.4 %. The amphibious wireless communication capability of the triboelectric sensor array enables touchless HRI in the air and underwater, even in the presence of obstacles, while the whole system realizes high- resolution tactile sensing. By naturally integrating remote contorl and tactile sensing, the proposed EM-Finger could pave the way for enhanced HRI in machine intelligence.

Vibration perception is essential for robotic sens- ing and dynamic control. Nevertheless, due to the rigorous demand for sensor conformability and stretchability, enabling soft robots with proprioceptive vibration sensing remains challenging. This paper proposes a novel liquid metal-based stretchable e-skin via a kirigami-inspired design to enable soft robot proprioceptive vibration sensing. The e-skin is fabricated into 0.1mm ultrathin thickness, ensuring its negligible influence on the overall stiffness of the soft robot. Moreover, the working mechanism of the e-skin is based on the ubiquitous triboelec- trification effect, which transduces mechanical stimuli without external power supply. To demonstrate the practicability of the e-skin, we built a soft gripper consisting of three soft robotic fingers with proprioceptive vibration sensing. Our experiment shows that the gripper can accurately distinguish the grain category (six grains with the same mass, 99.9% accuracy) and the packaging quality (100% accuracy) by simply shaking the gripped bottle. In summary, a soft robotic proprioceptive vibration sensing solution is proposed; it helps soft robots to have a more comprehensive awareness of their self-state and may inspire further research on soft robotics.

Abnormal vibration is a direct response to the mechanical defects of electrical equipment, and requires reliable vibration sensing for health condition evaluation in the associated system. The triboelectric nanogen- erator (TENG) triggered by random vibration to generate electrical energy/ signal while giving feedback on the vibration state, paving a promising way towards self-powered sensors. Here, an all-in-one sensing system config- ured with a vibration sensor demonstrates instantaneous discharge boosted TENG and IR wireless communication for vibration state online monitoring. The sandwich-structured TENG combined with mechanical switches can release the co-accumulated charges from dual triboelectric layers to yield giant instantaneous output power of 616 W, which is 106 times higher than that of the continuous discharge. Moreover, an IR LED as a transmitter driven by the TENG can form an all-in-one vibration sensor enabling wireless communication, where the sensor can be further integrated with repeaters and phones to establish a wireless vibration online monitoring system for vibration state visualization. This work presents a novel idea to implement high-power TENG with IR communication integration for in situ vibration online monitoring. Such a strategy is potentially available for distributed sensor construction towards abnormal signal monitoring that reflects the operating state of equipment.

In the era of 6G and the Internet of Things (IoT), massive amounts of data are produced by distributed sensors and transferred wirelessly between various smart devices. Meanwhile, the proportion of global energy expended on communications keeps increasing. A possible solution to reduce energy required for information transfer is harvesting pervasive mechanical energy. However, the popularization of so-far-realized visible-light-based self-powered optical wireless communications (OWC) systems is restricted by ambient light conditions and complex receiver designs. In this work, an infrared (IR)-based OWC system is proposed to leverage a triboelectric nanogenerator (TENG) to achieve information encoding and transmission using an IR signal that is robust against light interference. Specifically, the mechanical motion and mechanical structures of TENGs can be utilized to convey information and power the IR emitter. Moreover, the system supports diverse TENG structures and can accommodate different demands. Our research shows that self-powered IR-based OWC, with the merits of long transmission distances, high adaptability, and low cost, may significantly promote TENG-enabled OWC and pave the way for sustainable communications.