Toward in-sensor imaging classification enabled by on-chip all-optical modulation and photonic neural networks.

Area of Science:

• Photonics and optical engineering
• Artificial intelligence and machine learning
• Integrated circuit design

Background:

• Image sensors in autonomous driving and security generate excessive data due to sensor-processor separation.
• Current systems face challenges with redundant data transmission between sensory terminals and computing units.

Purpose of the Study:

• To develop an "in-sensor" imaging classification solution by integrating sensing and photonic neural networks.
• To overcome data redundancy issues in current image sensing technologies.
• To demonstrate a novel silicon photonic approach for efficient image processing.

Main Methods:

• Utilized silicon photonic integrated technology to combine sensing components with photonic neural networks.
• Developed a microring array to convert visible light to near-infrared signals for waveguide propagation.
• Employed an all-optical modulator based on a pn-doped microring resonator utilizing thermo-optic and plasma-dispersion effects.
• Implemented cascaded microrings for signal conversion and weight application, enabling dot product operations.

Main Results:

• Achieved a modulation depth of 15 dB by controlling thermo-optic and plasma-dispersion effects.
• Demonstrated dot product operations with an effective resolution of near 8 bits using cascaded microrings.
• Validated the in-sensor scheme on the Modified National Institute of Standards and Technology (MNIST) dataset with 96.82% recognition accuracy.

Conclusions:

• The proposed in-sensor imaging classification technology significantly reduces data redundancy.
• Silicon photonic integrated circuits offer a promising platform for advanced 'in-sensor' processing.
• This technology has potential applications in autonomous driving, security, and other fields requiring efficient image analysis.