Autonomous Causal Discovery: Evaluating LLMs' Priors and Constraint Strategies for Reliability

Area of Science:

• Artificial Intelligence
• Causal Inference
• Machine Learning

Background:

• Expert-guided Causal Structure Learning (CSL) enhances causal discovery accuracy using prior knowledge.
• Acquiring expert knowledge is challenging; Large Language Models (LLMs) offer an alternative, but their causal priors can be unreliable due to hallucinations.
• Existing methods lack systematic evaluation of LLM-derived constraints for CSL.

Purpose of the Study:

• To develop a framework for measuring the reliability and effectiveness of LLM-derived constraints in CSL.
• To systematically evaluate different types of structural constraints derived from LLMs.
• To propose an optimized CSL framework that effectively incorporates noisy LLM priors.

Main Methods:

• Introduced a structural constraint measurement framework defining constraint strength and quality.
• Evaluated five constraint categories: Edge Existence (EEC), Edge Forbidden (EFC), Path Existence (PEC), Path Forbidden (PFC), and Order Constraints (OC).
• Proposed a two-level CSL optimization framework partitioning the search space and refining structures with global path constraints.

Main Results:

• Edge Existence Constraints (EEC) show high strength but low quality from LLMs.
• Path Forbidden (PFC) and Order Constraints (OC) offer a balanced trade-off between search-space pruning and reliability.
• The proposed two-level CSL framework effectively incorporates noisy LLM priors, especially when expert knowledge is scarce.

Conclusions:

• LLM-derived constraints can be valuable for CSL, but their reliability must be carefully assessed.
• PFC and OC constraints provide a promising approach for leveraging LLM knowledge in CSL.
• The developed framework and optimization strategy enhance CSL performance in low-expert-knowledge settings.