NEOSTI - a neuromorphic electronic-opto spatial-temporal hybrid image sensor

Area of Science:

• Biomimetic Engineering
• Computer Vision
• Neuromorphic Computing

Background:

• Machine vision systems struggle with energy efficiency and processing power for large datasets.
• Human eyes achieve remarkable visual processing with minimal energy consumption.
• Biomimetic approaches offer potential solutions for advanced vision systems.

Purpose of the Study:

• To develop a compact, energy-efficient, eye-sized vision system inspired by biological eyes.
• To integrate optical and electronic processing for enhanced data handling in machine vision.
• To enable real-time image semantic processing using a Binary Neural Network.

Main Methods:

• Proposed the Neuromorphic Electronic-Opto Spatial Temporal Imager (NEOSTI), an integrated eye-sized vision system.
• Implemented a multi-stage processing architecture: pre-sensor optical, in-sensor nonlinear acquisition, and near-sensor electronic.
• Integrated a low-complexity Binary Neural Network for semantic information processing.

Main Results:

• NEOSTI demonstrates efficient data acquisition and processing in diverse lighting conditions.
• The system achieves parallel data computing capabilities during image sensing.
• Competitive performance was attained across various visual processing tasks.

Conclusions:

• NEOSTI represents a significant advancement in compact, low-power, high-performance vision systems.
• The biomimetic, multi-domain processing approach overcomes key limitations in current machine vision.
• This technology holds promise for energy-efficient and capable machine vision applications.