The exact form of the 'Ockham factor' in model selection
share
We unify the Bayesian and Frequentist justifications for model selection based upon maximizing the evidence, using a precise definition of model complexity which we call 'flexibility'. In the Gaussian linear model, flexibility is asymptotically equal to the Bayesian Information Criterion (BIC) penalty. But we argue against replacing flexibility with the BIC penalty. Instead, we advocate estimating the evidence directly, for which there now exists a wide range of approaches in the literature.
READ FULL TEXT