Causal Discovery with Reinforcement Learning

by   Shengyu Zhu, et al.

Discovering causal structure among a set of variables is a fundamental problem in many empirical sciences. Traditional score-based casual discovery methods rely on various local heuristics to search for a directly acyclic graph (DAG) according to a predefined score function. While these methods, e.g., greedy equivalence search (GES), may have attractive results with infinite samples and certain model assumptions, they are less satisfactory in practice due to finite data and possible violation of assumptions. Motivated by recent advances in neural combinatorial optimization, we propose to use reinforcement learning (RL) to search for the DAG with the best scoring. Our encoder-decoder model takes observable data as input and generates graph adjacency matrices that are used to compute corresponding rewards. The reward incorporates both the predefined score function and two penalty terms for enforcing acyclicity. In contrast with typical RL applications where the goal is to learn a policy, we use RL as a search strategy and our final output would be the graph, among all graphs generated during training, that achieves the best reward. We conduct experiments on both synthetic and real data, and show that the proposed approach not only has an improved search ability but also allows for a flexible score function under the acyclicity constraint.


page 1

page 2

page 3

page 4


Ordering-Based Causal Discovery with Reinforcement Learning

It is a long-standing question to discover causal relations among a set ...

Causal Discovery from Incomplete Data using An Encoder and Reinforcement Learning

Discovering causal structure among a set of variables is a fundamental p...

Discovering Dynamic Causal Space for DAG Structure Learning

Discovering causal structure from purely observational data (i.e., causa...

Masked Gradient-Based Causal Structure Learning

Learning causal graphical models based on directed acyclic graphs is an ...

KGS: Causal Discovery Using Knowledge-guided Greedy Equivalence Search

Learning causal relationships solely from observational data provides in...

Automatic Truss Design with Reinforcement Learning

Truss layout design, namely finding a lightweight truss layout satisfyin...

The RL Perceptron: Generalisation Dynamics of Policy Learning in High Dimensions

Reinforcement learning (RL) algorithms have proven transformative in a r...

Please sign up or login with your details

Forgot password? Click here to reset