Generation of ice states through deep reinforcement learning

Area of Science:

• Physics
• Computer Science
• Artificial Intelligence

Background:

• Generating ground states for models with topological constraints, like the square ice model, is computationally challenging.
• Traditional methods may struggle with complex constraint-preserving state generation.

Purpose of the Study:

• To develop and evaluate a deep reinforcement learning (DRL) framework for generating ground states of the square ice model.
• To investigate if a DRL agent can learn physical rules implicitly.

Main Methods:

• Training a machine agent using DRL to explore the physical environment and discover a policy for ground state generation.
• Analyzing the learned policy and state value function.
• Benchmarking the DRL agent as a Markov chain Monte Carlo sampler against a baseline loop algorithm.

Main Results:

• The trained DRL agent successfully generates ground states for the square ice model by proposing local moves.
• Analysis revealed the agent learned the ice rule and loop-closing condition without explicit programming.
• The DRL-based sampler demonstrated competitive performance against the baseline loop algorithm.

Conclusions:

• DRL provides a powerful framework for discovering policies to generate constraint-preserving states in physical models.
• This approach can be generalized to other complex systems with topological constraints.
• The AI agent implicitly learned fundamental physical principles governing the system.