Bioinspired Adaptive UV-NIR Cross-Band Convolutional Fusion for Motion and Shape Recognition in Adverse Environments

Area of Science:

• Neuromorphic engineering
• Artificial intelligence
• Materials science

Background:

• Current vision technologies lack integrated multispectral sensing, memory, and real-time processing.
• Emulating biological vision requires overcoming these technological limitations.

Purpose of the Study:

• To introduce the Bioinspired NeuroFusion Motion Intelligence (BNMI) platform.
• To enable reliable perception across a wide range of lighting conditions (dim light to glare).

Main Methods:

• Utilized dual-electrolyte-gated In2O3 transistors to emulate biological photoreceptors with stable, tunable photoresponses.
• Developed an all-integrated cross-band hardware convolution kernel (AICB-Conv) for direct UV/IR convolution, minimizing digitization delays.
• Implemented hardware-level fusion for feature-aware representations resilient to spectral degradation.

Main Results:

• Demonstrated efficient feature extraction and accurate motion decoding on M3SVD, MNIST, and custom video datasets.
• Achieved robust performance from dim light to glare, handling spectral degradation like UV overexposure or NIR attenuation.
• Showcased the BNMI platform's capability for real-time processing and in-sensor memory functions.

Conclusions:

• The BNMI platform represents a novel neuromorphic multispectral vision paradigm.
• This technology is suitable for applications in autonomous navigation, intelligent surveillance, and edge AI.
• The integrated approach overcomes key challenges in emulating biological vision.