A Heterogeneously Integrated Spiking Neuron Array for Multimode-Fused Perception and Object Classification.

Area of Science:

• Neuroscience and Robotics
• Materials Science and Engineering

Background:

• Multisensory integration in the somatosensory system is crucial for accurate object perception.
• Conventional metal-oxide-semiconductor technology faces challenges in integrating multisensory systems due to device complexity.

Purpose of the Study:

• To develop a compact multimode-fused spiking neuron (MFSN) for human-like multisensory perception.
• To address device integration and circuit complexity issues in conventional multisensory systems.

Main Methods:

• Heterogeneously integrated a pressure sensor and a NbO_x-based memristor into a single MFSN.
• Fused analog pressure and temperature data into a single spike train.
• Decoupled output frequencies and amplitudes to distinguish between pressure and temperature information.
• Fabricated a 3x3 MFSN array and simulated a larger array for tactile pattern recognition and object classification.

Main Results:

• The MFSN achieved efficient data compression and conversion of multisensory analog information into spike trains.
• Demonstrated successful distinction of pressure and temperature from fused spikes via output frequency and amplitude.
• Achieved enhanced tactile pattern recognition using a spiking neural network with the MFSN array.
• Validated the feasibility of MFSNs for classifying objects based on shape, temperature, and weight.

Conclusions:

• The developed MFSN offers a compact solution for multimodal tactile perception.
• Proof-of-concept MFSNs enable the construction of advanced multimodal sensory systems.
• This technology contributes to the development of highly intelligent robotics with enhanced sensory capabilities.