Multimodal In-Sensor Computing System Using Integrated Silicon Photonic Convolutional Processor

Area of Science:

• Photonics
• Spectroscopy
• Integrated Circuits

Background:

• Multimodal spectroscopic sensory systems generate large, complex data volumes.
• High communication bandwidth and power consumption are challenges for data transfer.
• In-sensor computing offers a solution to reduce data processing costs.

Purpose of the Study:

• To develop a photonic multimodal in-sensor computing system.
• To integrate photonic sensors with a silicon photonic convolutional processor.
• To demonstrate in situ processing of spectroscopic sensory data.

Main Methods:

• Utilized a microring resonator crossbar array as a photonic processor.
• Implemented convolutional operations with 5-bit accuracy for image edge detection.
• Integrated the processor with a photonic spectroscopic sensor for multimodal data analysis.

Main Results:

• Achieved 5-bit accuracy in convolutional operations via image edge detection.
• Demonstrated in situ processing of multimodal spectroscopic data.
• Attained 97.58% classification accuracy across 45 classes for protein species detection.

Conclusions:

• The developed system enables in-sensor computing for photonic multimodal spectroscopic sensors.
• Integration of photonic processors and sensors enhances edge data processing capabilities.
• This approach significantly reduces communication costs associated with sensory data transfer.