In-sensor multilevel image adjustment for high-clarity contour extraction using adjustable synaptic phototransistors

Area of Science:

• Robotics and Computer Vision
• Materials Science and Engineering
• Neuro-inspired Computing

Background:

• Traditional robotic vision systems are computationally intensive, requiring high data transmission rates that compromise energy efficiency and speed.
• Data compaction via contour extraction offers a solution by focusing on object outlines and reducing background data.
• Existing methods lack efficient in-sensor processing for optimal contour extraction under varying lighting conditions.

Purpose of the Study:

• To develop an in-sensor multilevel image adjustment method for enhanced robotic vision.
• To enable high-clarity contour extraction by optimizing image brightness and contrast within the sensor.
• To improve energy efficiency and processing speed in robotic vision applications.

Main Methods:

• Introduced adjustable synaptic phototransistors capable of in-sensor multilevel image adjustment.
• Emulated dopamine-mediated neuronal excitability to regulate photocurrent accumulation via electrostatic gating.
• Utilized excitatory and inhibitory modes to enhance visibility in dim and bright regions for contour extraction.

Main Results:

• Demonstrated capture of well-defined images with optimal brightness and contrast for high-clarity contour extraction.
• Achieved improved object detection accuracy and intersection over union (IoU) in evaluations using road images.
• Showcased significant compression of data volume, leading to enhanced processing efficiency.

Conclusions:

• The developed in-sensor image adjustment method effectively facilitates distinct contour extraction and high-accuracy semantic segmentation.
• The adjustable synaptic phototransistor technology offers a promising solution for energy-efficient and high-speed robotic vision.
• This approach advances neuro-inspired computing for practical applications in autonomous systems.