Related Experiment Video
Updated: Dec 27, 2025
13:19
Deep Neural Networks for Image-Based Dietary Assessment
Published on: March 13, 2021
9.8K
Ultrafast machine vision with 2D material neural network image sensors
Lukas Mennel1, Joanna Symonowicz2, Stefan Wachter2
1Institute of Photonics, Vienna University of Technology, Vienna, Austria. lukas.mennel@tuwien.ac.at.
Nature
|March 6, 2020
Summary
Researchers developed a novel image sensor that acts as an artificial neural network (ANN), enabling simultaneous image sensing and processing. This breakthrough reduces latency and power consumption in machine vision systems.
Area of Science:
- Optoelectronics
- Artificial Intelligence
- Materials Science
Background:
- Machine vision systems commonly use frame-based cameras and artificial neural networks (ANNs), leading to high data redundancy, low frame rates, and significant power consumption.
- Existing visual data preprocessing techniques aim to improve the efficiency of ANNs but do not address the fundamental latency and power issues of the sensor-to-processor pipeline.
Purpose of the Study:
- To demonstrate an image sensor capable of performing artificial neural network (ANN) computations, thereby enabling simultaneous image sensing and processing.
- To develop a novel device that integrates sensing and processing to overcome the limitations of conventional machine vision systems.
Main Methods:
- Utilized a reconfigurable two-dimensional (2D) semiconductor photodiode array as the core of the artificial neural network (ANN).
- Implemented a continuously tunable photoresponsivity matrix to store and adjust synaptic weights within the sensor.
- Demonstrated both supervised and unsupervised learning paradigms on the integrated sensor.
Main Results:
- The developed image sensor functions as an artificial neural network (ANN), performing sensing and processing concurrently with minimal latency.
- The device successfully classified and encoded optically projected images with a high throughput of 20 million bins per second.
- Achieved efficient image processing directly at the sensor level, bypassing traditional data conversion and transfer bottlenecks.
Conclusions:
- The novel image sensor architecture effectively merges sensing and artificial neural network (ANN) processing, offering a significant advancement in machine vision.
- This integrated approach promises to enhance the speed and energy efficiency of intelligent systems like autonomous vehicles and robotics.
- The demonstrated reconfigurable photodiode array opens new avenues for low-latency, high-throughput optical data processing applications.
