Optical neural ordinary differential equations.

Area of Science:

• Integrated photonics
• Artificial intelligence hardware
• Neural network architectures

Background:

• Increasing layers in on-chip photonic neural networks (PNNs) enhances model performance but increases chip area.
• Cascading hidden layers leads to significant area occupancy challenges for complex PNNs.

Purpose of the Study:

• To propose a novel architecture, optical neural ordinary differential equations (ON-ODEs), to reduce the physical footprint of PNNs.
• To enable high-performance PNNs with reduced area occupancy through continuous layer parameterization.

Main Methods:

• Developed the ON-ODE architecture, integrating PNNs with photonic integrators and optical feedback loops.
• Configured ON-ODEs to emulate residual neural networks (ResNets) and recurrent neural networks.
• Utilized optical ODE solvers to parameterize continuous dynamics of hidden layers.

Main Results:

• A single-layer ON-ODE with an interference-based optoelectronic nonlinear hidden layer matched the accuracy of two-layer optical ResNets in image classification.
• The ON-ODE architecture improved classification accuracy for diffraction-based all-optical linear hidden layers.
• Demonstrated high accuracy in trajectory prediction tasks by leveraging the time-dependent dynamics of ON-ODEs.

Conclusions:

• The ON-ODE architecture offers a significant reduction in chip area for PNNs without compromising model performance.
• ON-ODEs provide a versatile platform for implementing advanced neural network functionalities on-chip.
• This approach advances the development of efficient and powerful integrated photonic AI accelerators.