Belief Propagation, Bethe Approximation and Polynomials

08/08/2017
by   Damian Straszak, et al.
0

Factor graphs are important models for succinctly representing probability distributions in machine learning, coding theory, and statistical physics. Several computational problems, such as computing marginals and partition functions, arise naturally when working with factor graphs. Belief propagation is a widely deployed iterative method for solving these problems. However, despite its significant empirical success, not much is known about the correctness and efficiency of belief propagation. Bethe approximation is an optimization-based framework for approximating partition functions. While it is known that the stationary points of the Bethe approximation coincide with the fixed points of belief propagation, in general, the relation between the Bethe approximation and the partition function is not well understood. It has been observed that for a few classes of factor graphs, the Bethe approximation always gives a lower bound to the partition function, which distinguishes them from the general case, where neither a lower bound, nor an upper bound holds universally. This has been rigorously proved for permanents and for attractive graphical models. Here we consider bipartite normal factor graphs and show that if the local constraints satisfy a certain analytic property, the Bethe approximation is a lower bound to the partition function. We arrive at this result by viewing factor graphs through the lens of polynomials. In this process, we reformulate the Bethe approximation as a polynomial optimization problem. Our sufficient condition for the lower bound property to hold is inspired by recent developments in the theory of real stable polynomials. We believe that this way of viewing factor graphs and its connection to real stability might lead to a better understanding of belief propagation and factor graphs in general.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
11/12/2018

Gauges, Loops, and Polynomials for Partition Functions of Graphical Models

We suggest a new methodology for analysis and approximate computations o...
research
02/13/2020

Sensitivity of Wardrop Equilibria: Revisited

For single-commodity networks, the increase of the price of anarchy is b...
research
05/01/2023

Approximating submodular k-partition via principal partition sequence

In submodular k-partition, the input is a non-negative submodular functi...
research
05/16/2018

Max-Product for Maximum Weight Matching - Revisited

We focus on belief propagation for the assignment problem, also known as...
research
05/01/2021

The complexity of approximating the complex-valued Ising model on bounded degree graphs

We study the complexity of approximating the partition function Z_Ising(...
research
02/14/2012

What Cannot be Learned with Bethe Approximations

We address the problem of learning the parameters in graphical models wh...
research
03/23/2022

Factor Graphs for Quantum Information Processing

[...] In this thesis, we are interested in generalizing factor graphs an...

Please sign up or login with your details

Forgot password? Click here to reset