Occam Factor for Gaussian Models With Unknown Variance Structure

by   Zachary M. Pisano, et al.

We discuss model selection to determine whether the variance-covariance matrix of a multivariate Gaussian model with known mean should be considered to be a constant diagonal, a non-constant diagonal, or an arbitrary positive definite matrix. Of particular interest is the relationship between Bayesian evidence and the flexibility penalty due to Priebe and Rougier. For the case of an exponential family in canonical form equipped with a conjugate prior for the canonical parameter, flexibility may be exactly decomposed into the usual BIC likelihood penalty and a O_p(1) term, the latter of which we explicitly compute. We also investigate the asymptotics of Bayes factors for linearly nested canonical exponential families equipped with conjugate priors; in particular, we find the exact rates at which Bayes factors correctly diverge in favor of the correct model: linearly and logarithmically in the number of observations when the full and nested models are true, respectively. Such theoretical considerations for the general case permit us to fully express the asymptotic behavior of flexibility and Bayes factors for the variance-covariance structure selection problem when we assume that the prior for the model precision is a member of the gamma/Wishart family of distributions or is uninformative. Simulations demonstrate evidence's immediate and superior performance in model selection compared to approximate criteria such as the BIC. We extend the framework to the multivariate Gaussian linear model with three data-driven examples.


page 1

page 2

page 3

page 4


Conjugate Bayesian analysis of compound-symmetric Gaussian models

We discuss Bayesian inference for a known-mean Gaussian model with a com...

The exact form of the 'Ockham factor' in model selection

We unify the Bayesian and Frequentist justifications for model selection...

Occam Factor for Random Graphs: Erdös-Rényi, Independent Edge, and a Uniparametric Stochastic Blockmodel

We investigate the evidence/flexibility (i.e., "Occam") paradigm and dem...

Graphical Evidence

Marginal likelihood, also known as model evidence, is a fundamental quan...

Extreme data compression for Bayesian model comparison

We develop extreme data compression for use in Bayesian model comparison...

Reconciling the Bayes Factor and Likelihood Ratio for Two Non-Nested Model Selection Problems

In statistics, there are a variety of methods for performing model selec...

Quantifying the weight of fingerprint evidence using an ROC-based Approximate Bayesian Computation algorithm

The Bayes factor has been advocated to quantify the weight of forensic e...

Please sign up or login with your details

Forgot password? Click here to reset