2D Time-Stretching Anisotropic Synapse Realizing In-Sensor Intensity-Spanning Visual Feature Fusion

Area of Science:

• Materials Science
• Neuromorphic Engineering
• Computer Vision

Background:

• Conventional imaging struggles with high-dynamic-range (HDR) environments due to latency and privacy issues with multi-frame processing.
• Real-time and edge-level intelligent vision require efficient, low-latency solutions for processing extreme brightness variations.

Purpose of the Study:

• To develop an in-sensor solution for feature fusion in HDR visual environments using a single image frame.
• To enable single-shot visual learning across extreme brightness domains for neuromorphic preprocessing.

Main Methods:

• Development of a 2D Time-Stretching Anisotropic Synapse (2D TSAS) utilizing NbOI2 material properties.
• Integration of in-plane anisotropy for polarization-resolved optical responses and time-stretching photoresponse for multi-channel transition-relaxation.
• Construction of a neuromorphic preprocessing strategy for single-shot visual learning.

Main Results:

• The 2D TSAS enables direct encoding and temporal integration of spatial-polarization and luminance features.
• Achieved accelerated model convergence with minimal training loss in neuromorphic preprocessing.
• Demonstrated high recognition accuracies of ≈95.41% on NWPU-RESISC45 and ≈95.39% on MNIST.

Conclusions:

• The 2D TSAS offers a compact and efficient solution for contrast-adaptive intelligent vision in complex environments.
• This in-sensor approach overcomes limitations of traditional HDR imaging and cloud-based processing.
• Paves the way for advanced real-time intelligent vision applications in challenging lighting conditions.