Causal discovery by continuous optimization with weighted superstructure

Recently, score-based methods that frame causal discovery as a continuous optimization problem have achieved significant performance gains. However, their performance often degrades in scenarios characterized by high dimensionality, limited samples, and heterogeneous noise. It has been empirically observed that constraint-based methods generally perform better in learning causal structures under heterogeneous noise with fewer samples than the continuous optimization based methods, which motivates us to improve continuous optimization based causal discovery by introducing and exploiting conditional independence (CI) information. To obtain reliable CI information, we use low-order (0-order and 1-order) CI tests to construct a weighted superstructure from the observed data. Then, we propose the weighted CI constraints into continuous optimization, where the constraints can be easily obtained from the weighted superstructure. Theoretically, we provide a convergence guarantee for our constrained optimization framework with the CI constraints. Extensive experiments on synthetic and real-world datasets demonstrate that our proposal effectively improves the performance of existing continuous optimization methods in causal discovery, particularly in the low-sample regime. The source code and data are available at https://github.com/Mjchen22/WIC.