DeepAI AI Chat
Log In Sign Up

Convergence of uncertainty estimates in Ensemble and Bayesian sparse model discovery

by   L. Mars Gao, et al.

Sparse model identification enables nonlinear dynamical system discovery from data. However, the control of false discoveries for sparse model identification is challenging, especially in the low-data and high-noise limit. In this paper, we perform a theoretical study on ensemble sparse model discovery, which shows empirical success in terms of accuracy and robustness to noise. In particular, we analyse the bootstrapping-based sequential thresholding least-squares estimator. We show that this bootstrapping-based ensembling technique can perform a provably correct variable selection procedure with an exponential convergence rate of the error rate. In addition, we show that the ensemble sparse model discovery method can perform computationally efficient uncertainty estimation, compared to expensive Bayesian uncertainty quantification methods via MCMC. We demonstrate the convergence properties and connection to uncertainty quantification in various numerical studies on synthetic sparse linear regression and sparse model discovery. The experiments on sparse linear regression support that the bootstrapping-based sequential thresholding least-squares method has better performance for sparse variable selection compared to LASSO, thresholding least-squares, and bootstrapping-based LASSO. In the sparse model discovery experiment, we show that the bootstrapping-based sequential thresholding least-squares method can provide valid uncertainty quantification, converging to a delta measure centered around the true value with increased sample sizes. Finally, we highlight the improved robustness to hyperparameter selection under shifting noise and sparsity levels of the bootstrapping-based sequential thresholding least-squares method compared to other sparse regression methods.


page 1

page 2

page 3

page 4


Ensemble-SINDy: Robust sparse model discovery in the low-data, high-noise limit, with active learning and control

Sparse model identification enables the discovery of nonlinear dynamical...

Spike-and-Slab Group Lassos for Grouped Regression and Sparse Generalized Additive Models

We introduce the spike-and-slab group lasso (SSGL) for Bayesian estimati...

Nonparametric Empirical Bayes Estimation and Testing for Sparse and Heteroscedastic Signals

Large-scale modern data often involves estimation and testing for high-d...

Quantile universal threshold: model selection at the detection edge for high-dimensional linear regression

To estimate a sparse linear model from data with Gaussian noise, consili...

Bayesian autoencoders for data-driven discovery of coordinates, governing equations and fundamental constants

Recent progress in autoencoder-based sparse identification of nonlinear ...

Sparse Methods for Automatic Relevance Determination

This work considers methods for imposing sparsity in Bayesian regression...

Least Absolute Gradient Selector: Statistical Regression via Pseudo-Hard Thresholding

Variable selection in linear models plays a pivotal role in modern stati...