PPO-GPR: A Custom Proximal Policy Optimization Tool for Active Reinforcement Learning

Area of Science:

• Computational Material Science
• Machine Learning in Chemistry
• Reinforcement Learning for Scientific Discovery

Background:

• Expensive and time-consuming data acquisition hinders progress in material science.
• Predictive modeling requires strategic data selection to maximize efficiency.
• Active learning offers a promising approach to optimize data acquisition.

Purpose of the Study:

• To develop a novel active learning framework integrating Proximal Policy Optimization (PPO) with Gaussian Process Regression (GPR).
• To strategically select informative data points for enhanced predictive modeling in material science.
• To accelerate the discovery of new materials and optimize gas separation processes.

Main Methods:

• Integration of PPO with GPR for guided data acquisition.
• Development of a custom Gymnasium environment for PPO agent training.
• Utilizing R-squared score for GPR performance evaluation and action masking to prevent data redundancy.

Main Results:

• Achieved 77-86% data savings compared to full GCMC grids for predicting methane selectivity in CuBTC and IRMOF-1.
• Successfully queried only ~14-23% of the candidate pool while maintaining high predictive accuracy (R-squared, MAE, RMSE).
• Demonstrated stable convergence of the clipped-update PPO policy by focusing on critical pressure-temperature-composition regions.

Conclusions:

• The PPO-GPR framework enables highly efficient data selection for predictive modeling in material science.
• This approach significantly accelerates material discovery and optimizes gas separation processes.
• Combines reinforcement learning and regression models for effective scientific acceleration.