Gradient-based data and parameter dimension reduction for Bayesian models: an information theoretic perspective

by   Ricardo Baptista, et al.

We consider the problem of reducing the dimensions of parameters and data in non-Gaussian Bayesian inference problems. Our goal is to identify an "informed" subspace of the parameters and an "informative" subspace of the data so that a high-dimensional inference problem can be approximately reformulated in low-to-moderate dimensions, thereby improving the computational efficiency of many inference techniques. To do so, we exploit gradient evaluations of the log-likelihood function. Furthermore, we use an information-theoretic analysis to derive a bound on the posterior error due to parameter and data dimension reduction. This bound relies on logarithmic Sobolev inequalities, and it reveals the appropriate dimensions of the reduced variables. We compare our method with classical dimension reduction techniques, such as principal component analysis and canonical correlation analysis, on applications ranging from mechanics to image processing.


page 16

page 25

page 26

page 27


Certified dimension reduction in nonlinear Bayesian inverse problems

We propose a dimension reduction technique for Bayesian inverse problems...

Data-Free Likelihood-Informed Dimension Reduction of Bayesian Inverse Problems

Identifying a low-dimensional informed parameter subspace offers a viabl...

Model Reduction of Linear Dynamical Systems via Balancing for Bayesian Inference

We consider the Bayesian approach to the linear Gaussian inference probl...

Adaptive Dimension Reduction to Accelerate Infinite-Dimensional Geometric Markov Chain Monte Carlo

Bayesian inverse problems highly rely on efficient and effective inferen...

Principal Loading Analysis

This paper proposes a tool for dimension reduction where the dimension o...

Accelerating Derivative-Free Optimization with Dimension Reduction and Hyperparameter Learning

We consider convex, black-box objective functions with additive or multi...

Please sign up or login with your details

Forgot password? Click here to reset