Convergence of uncertainty estimates in Ensemble and Bayesian sparse model discovery

01/30/2023
by   L. Mars Gao, et al.
3

Sparse model identification enables nonlinear dynamical system discovery from data. However, the control of false discoveries for sparse model identification is challenging, especially in the low-data and high-noise limit. In this paper, we perform a theoretical study on ensemble sparse model discovery, which shows empirical success in terms of accuracy and robustness to noise. In particular, we analyse the bootstrapping-based sequential thresholding least-squares estimator. We show that this bootstrapping-based ensembling technique can perform a provably correct variable selection procedure with an exponential convergence rate of the error rate. In addition, we show that the ensemble sparse model discovery method can perform computationally efficient uncertainty estimation, compared to expensive Bayesian uncertainty quantification methods via MCMC. We demonstrate the convergence properties and connection to uncertainty quantification in various numerical studies on synthetic sparse linear regression and sparse model discovery. The experiments on sparse linear regression support that the bootstrapping-based sequential thresholding least-squares method has better performance for sparse variable selection compared to LASSO, thresholding least-squares, and bootstrapping-based LASSO. In the sparse model discovery experiment, we show that the bootstrapping-based sequential thresholding least-squares method can provide valid uncertainty quantification, converging to a delta measure centered around the true value with increased sample sizes. Finally, we highlight the improved robustness to hyperparameter selection under shifting noise and sparsity levels of the bootstrapping-based sequential thresholding least-squares method compared to other sparse regression methods.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
11/22/2021

Ensemble-SINDy: Robust sparse model discovery in the low-data, high-noise limit, with active learning and control

Sparse model identification enables the discovery of nonlinear dynamical...
research
03/05/2019

Spike-and-Slab Group Lassos for Grouped Regression and Sparse Generalized Additive Models

We introduce the spike-and-slab group lasso (SSGL) for Bayesian estimati...
research
06/16/2021

Nonparametric Empirical Bayes Estimation and Testing for Sparse and Heteroscedastic Signals

Large-scale modern data often involves estimation and testing for high-d...
research
12/05/2014

Quantile universal threshold: model selection at the detection edge for high-dimensional linear regression

To estimate a sparse linear model from data with Gaussian noise, consili...
research
11/19/2022

Bayesian autoencoders for data-driven discovery of coordinates, governing equations and fundamental constants

Recent progress in autoencoder-based sparse identification of nonlinear ...
research
05/18/2020

Sparse Methods for Automatic Relevance Determination

This work considers methods for imposing sparsity in Bayesian regression...
research
04/11/2012

Least Absolute Gradient Selector: Statistical Regression via Pseudo-Hard Thresholding

Variable selection in linear models plays a pivotal role in modern stati...

Please sign up or login with your details

Forgot password? Click here to reset