Prequential MDL for Causal Structure Learning with Neural Networks

by   Jörg Bornschein, et al.

Learning the structure of Bayesian networks and causal relationships from observations is a common goal in several areas of science and technology. We show that the prequential minimum description length principle (MDL) can be used to derive a practical scoring function for Bayesian networks when flexible and overparametrized neural networks are used to model the conditional probability distributions between observed variables. MDL represents an embodiment of Occam's Razor and we obtain plausible and parsimonious graph structures without relying on sparsity inducing priors or other regularizers which must be tuned. Empirically we demonstrate competitive results on synthetic and real-world data. The score often recovers the correct structure even in the presence of strongly nonlinear relationships between variables; a scenario were prior approaches struggle and usually fail. Furthermore we discuss how the the prequential score relates to recent work that infers causal structure from the speed of adaptation when the observations come from a source undergoing distributional shift.


A Bayesian Approach to Learning Bayesian Networks with Local Structure

Recently several researchers have investigated techniques for using data...

Bayesian network structure learning with causal effects in the presence of latent variables

Latent variables may lead to spurious relationships that can be misinter...

Network Learning Approaches to study World Happiness

The United Nations in its 2011 resolution declared the pursuit of happin...

A Bayesian Approach for Inferring Local Causal Structure in Gene Regulatory Networks

Gene regulatory networks play a crucial role in controlling an organism'...

Neural Bayesian Network Understudy

Bayesian Networks may be appealing for clinical decision-making due to t...

Bidirectional Inference Networks: A Class of Deep Bayesian Networks for Health Profiling

We consider the problem of inferring the values of an arbitrary set of v...

Please sign up or login with your details

Forgot password? Click here to reset