In-sensor compressing via programmable optoelectronic sensors based on van der Waals heterostructures for intelligent...

Area of Science:

• Optoelectronics
• Materials Science
• Computer Vision

Background:

• Intelligent machine vision requires efficient capture of multidimensional signals.
• In-sensor computing offers reduced data transfer but limited temporal/spectral data compression.
• Existing methods struggle with efficient compression of complex visual data.

Purpose of the Study:

• To demonstrate a programmable 2D heterostructure-based optoelectronic sensor for in-sensor data compression.
• To integrate sensing, memory, and computation within a single device.
• To evaluate the sensor's performance in compressing dynamic videos and spectral data.

Main Methods:

• Development of a programmable two-dimensional (2D) heterostructure-based optoelectronic sensor.
• Integration of sensing, memory, and computation functionalities.
• In-device snapshot compression of dynamic videos and 3D spectral data.
• Evaluation of reconstruction quality using peak signal-to-noise ratio (PSNR).
• Classification of compressed video data using in-sensor convolution.

Main Results:

• Achieved an 8:1 compression ratio for dynamic videos and 3D spectral data.
• Demonstrated comparable reconstruction quality (15.81 dB) to software methods (16.21 dB).
• Preserved semantic information in compressed videos for accurate classification.
• Attained high classification accuracy (93.18%) for compressed videos via in-sensor convolution.

Conclusions:

• The developed 2D optoelectronic sensor enables efficient in-device data compression.
• This technology supports the development of edge-based intelligent vision systems.
• The sensor's capabilities advance efficient processing of spectral and temporal visual information.