Brain-Inspired Reservoir Computing with Dynamic Memristors for Audio-Visual Emotion Recognition.

Area of Science:

• Neuromorphic Engineering
• Artificial Intelligence
• Materials Science

Background:

• Edge AI and multimodal perception systems face computational bottlenecks and high training costs due to von Neumann architecture limitations.
• Emerging device technologies and novel computing architectures are crucial for improving efficiency and reducing energy consumption in artificial perception.

Purpose of the Study:

• To develop a brain-inspired reservoir computing (RC) framework using dynamic memristors for efficient audio-visual emotion recognition.
• To address the limitations of current architectures for edge AI applications.

Main Methods:

• Utilized Ag/WOx/ITO dynamic memristors exhibiting biological synaptic functionalities as physical reservoirs.
• Employed the memristor's transient response and nonlinear mapping to transform spatiotemporal audio-visual signals into separable high-dimensional reservoir states using 4-bit pulse sequences.

Main Results:

• Achieved 95.46% accuracy for speech-based emotion recognition and 91.08% for facial expression recognition.
• Demonstrated ultra-low energy consumption of approximately 3.9 nJ per pulse operation.

Conclusions:

• The developed memristor-based RC framework offers a viable hardware paradigm for low-power, real-time audio-visual perception on edge devices.
• This approach paves the way for advanced neuromorphic computing and more efficient artificial perception systems.