Graph-Attention-Based Casual Discovery With Trust Region-Navigated Clipping Policy Optimization

Area of Science:

• Causal inference
• Machine learning
• Data science

Background:

• Causal discovery is crucial in empirical sciences but faces challenges with unoriented edges and latent assumptions.
• Conventional methods struggle with these issues, prompting the use of reinforcement learning (RL).
• Existing RL methods like REINFORCE have limitations in stability and convergence.

Purpose of the Study:

• To develop a more robust and efficient reinforcement learning (RL) procedure for causal discovery.
• To address the limitations of existing RL algorithms (REINFORCE, PPO) in causal discovery tasks.
• To improve the encoding of variables for enhanced causal structure learning.

Main Methods:

• Proposed a trust region-navigated clipping policy optimization method for improved RL stability and efficiency.
• Introduced a refined graph attention encoder, SDGAT, for efficient variable encoding without prior neighborhood information.
• Implemented a prioritized sampling-guided REINFORCE for comparison.

Main Results:

• The proposed method demonstrated superior search efficiency and steadiness in policy optimization over REINFORCE and PPO.
• SDGAT effectively captured more feature information for variable encoding.
• The overall approach outperformed previous RL methods on synthetic and benchmark datasets.

Conclusions:

• The novel RL-based method offers significant improvements in causal discovery robustness and efficiency.
• The combination of trust region-navigated clipping policy optimization and SDGAT addresses key limitations in prior RL approaches.
• This work advances the field of causal discovery with a more reliable and performant methodology.