Hundred-layer photonic deep learning.

Area of Science:

• Photonics and Artificial Intelligence
• Integrated Photonic Circuits
• Deep Learning Hardware

Background:

• Analog nature of photonic computing limits neural network depth due to error accumulation.
• Current optical neural networks are restricted to approximately ten layers, hindering advanced AI capabilities like large language models (LLMs).
• Propagation redundancies in optical systems contribute significantly to accumulated errors.

Purpose of the Study:

• To overcome the depth limitations of optical neural networks.
• To develop an error-tolerant photonic computing chip for deep learning.
• To enable energy-efficient advanced AI models on analog hardware.

Main Methods:

• Introduced on-chip perturbations to decouple computational correlations and eliminate propagation redundancies.
• Developed a single-layer photonic computing (SLiM) chip architecture.
• Experimentally constructed deep neural networks, including LLMs, utilizing the SLiM chip.

Main Results:

• The SLiM chip demonstrates error tolerance, constraining error rates across over 200 layers.
• Extended spatial depth from millimeters to hundred-meter scale, facilitating 3D chip clustering.
• Achieved comparable performance to ideal simulations for a 100-layer image classification network and LLMs for text and image generation at a 10-GHz data rate.

Conclusions:

• The error-tolerant SLiM chip removes depth limitations in optical neural networks.
• This advancement enables state-of-the-art deep learning models on efficient analog computing hardware.
• Paves the way for more powerful and energy-efficient AI systems.