Is Deep Learning an RG Flow?

by   Ellen de Mello Koch, et al.

Although there has been a rapid development of practical applications, theoretical explanations of deep learning are in their infancy. A possible starting point suggests that deep learning performs a sophisticated coarse graining. Coarse graining is the foundation of the renormalization group (RG), which provides a systematic construction of the theory of large scales starting from an underlying microscopic theory. In this way RG can be interpreted as providing a mechanism to explain the emergence of large scale structure, which is directly relevant to deep learning. We pursue the possibility that RG may provide a useful framework within which to pursue a theoretical explanation of deep learning. A statistical mechanics model for a magnet, the Ising model, is used to train an unsupervised RBM. The patterns generated by the trained RBM are compared to the configurations generated through a RG treatment of the Ising model. We argue that correlation functions between hidden and visible neurons are capable of diagnosing RG-like coarse graining. Numerical experiments show the presence of RG-like patterns in correlators computed using the trained RBMs. The observables we consider are also able to exhibit important differences between RG and deep learning.


page 9

page 10


Short sighted deep learning

A theory explaining how deep learning works is yet to be developed. Prev...

A Deep Dive into the Connections Between the Renormalization Group and Deep Learning in the Ising Model

The renormalization group (RG) is an essential technique in statistical ...

An exact mapping between the Variational Renormalization Group and Deep Learning

Deep learning is a broad set of techniques that uses multiple layers of ...

Neurashed: A Phenomenological Model for Imitating Deep Learning Training

To advance deep learning methodologies in the next decade, a theoretical...

Astronomia ex machina: a history, primer, and outlook on neural networks in astronomy

In recent years, deep learning has infiltrated every field it has touche...

Scale-invariant Feature Extraction of Neural Network and Renormalization Group Flow

Theoretical understanding of how deep neural network (DNN) extracts feat...

Graphic displays of MLB pitching mechanics and its evolutions in PITCHf/x data

Systemic and idiosyncratic patterns in pitching mechanics of 24 top star...

Please sign up or login with your details

Forgot password? Click here to reset