Adaptive optics control with multi-agent model-free reinforcement learning.

Area of Science:

• Astronomy and Astrophysics
• Control Systems Engineering
• Artificial Intelligence

Background:

• Adaptive optics (AO) systems correct atmospheric distortions in real-time for clearer astronomical images.
• Traditional AO controllers often rely on pre-existing atmospheric models or simpler control strategies.
• Reinforcement learning (RL) offers a data-driven approach to control but faces challenges in complex systems like AO.

Purpose of the Study:

• To formulate closed-loop adaptive optics control as a multi-agent reinforcement learning (MARL) problem.
• To develop an AO controller capable of learning non-linear policies without prior atmospheric knowledge.
• To address challenges in applying RL to AO, including noise mitigation and performance evaluation.

Main Methods:

• Formulation of AO control as a MARL problem.
• Combination of model-free MARL with an autoencoder neural network for noise reduction.
• Extension of error budget analysis to incorporate RL controllers.

Main Results:

• Experimental validation on an 8m telescope with a 40x40 Shack-Hartmann system.
• Significant performance increase compared to integrator baseline controllers.
• Comparable performance to model-based predictive control (LQG) with perfect atmospheric knowledge.

Conclusions:

• The developed MARL-based AO controller learns effective non-linear control policies.
• The approach demonstrates robustness and improved performance in realistic astronomical conditions.
• Error budget analysis indicates compensation for bandwidth limitations and aliasing mitigation by the RL controller.