Streamlined optical training of large-scale modern deep learning architectures with direct feedback alignment

Area of Science:

• Artificial Intelligence
• Optical Computing
• Deep Learning Hardware

Background:

• Deep learning heavily relies on electronic hardware accelerators, limiting performance and energy efficiency.
• Photonic approaches show promise for inference but are limited in complexity.
• Training deep neural networks via backpropagation is a major computational bottleneck.

Purpose of the Study:

• To experimentally implement a scalable training algorithm, direct feedback alignment, on a hybrid electronic-photonic platform.
• To leverage optical processing for large-scale matrix multiplications central to deep learning training.
• To demonstrate the feasibility of optical training for complex, large-scale neural network architectures.

Main Methods:

• Experimental implementation of direct feedback alignment on a hybrid electronic-photonic system.
• Utilizing an optical processing unit for random matrix multiplications.
• Training deep learning models including Transformers with over 1 billion parameters.

Main Results:

• Successful optical training of deep learning architectures, including Transformers.
• Achieved strong performance on language, vision, and generative tasks.
• Demonstrated potential for faster training times in ultra-deep and wide neural networks.

Conclusions:

• The hybrid electronic-photonic approach offers a promising route for training large-scale neural networks.
• This method can potentially overcome the limitations of traditional von Neumann architectures for artificial intelligence.
• Opens new possibilities for advancing artificial intelligence capabilities beyond current hardware constraints.