Model-free optical processors using in situ reinforcement learning with proximal policy optimization.

Area of Science:

• Computational physics
• Optical engineering
• Machine learning

Background:

• Diffractive optical networks offer potential for high-speed, energy-efficient computing.
• Optimizing diffractive layers is challenging due to hardware imperfections and noise.
• Existing in situ methods suffer from slow convergence and instability.

Purpose of the Study:

• To introduce a novel reinforcement learning approach for in situ training of diffractive optical processors.
• To overcome limitations of existing optimization methods for physical optical systems.

Main Methods:

• Utilized Proximal Policy Optimization (PPO), a model-free reinforcement learning algorithm.
• Applied PPO for in situ training directly on the physical diffractive optical system.
• Leveraged efficient reuse of measurement data and constrained policy updates for stability.

Main Results:

• Demonstrated faster convergence and improved performance across various tasks: energy focusing, image generation, aberration correction, and classification.
• Validated the approach through both simulations and physical experiments.
• Showcased the method's ability to handle unknown real-world imperfections.

Conclusions:

• The model-free reinforcement learning approach provides a scalable framework for training complex optical systems.
• This method enables faster, more accurate, and robust in situ optimization under realistic experimental constraints.
• Eliminates the need for explicit system modeling or prior knowledge.