Variational Multiscale Nonparametric Regression: Algorithms and Implementation

by   Miguel del Álamo, et al.

Many modern statistically efficient methods come with tremendous computational challenges, often leading to large-scale optimisation problems. In this work, we examine such computational issues for recently developed estimation methods in nonparametric regression with a specific view on image denoising. We consider in particular certain variational multiscale estimators which are statistically optimal in minimax sense, yet computationally intensive. Such an estimator is computed as the minimiser of a smoothness functional (e.g., TV norm) over the class of all estimators such that none of its coefficients with respect to a given multiscale dictionary is statistically significant. The so obtained multiscale Nemirowski-Dantzig estimator (MIND) can incorporate any convex smoothness functional and combine it with a proper dictionary including wavelets, curvelets and shearlets. The computation of MIND in general requires to solve a high-dimensional constrained convex optimisation problem with a specific structure of the constraints induced by the statistical multiscale testing criterion. To solve this explicitly, we discuss three different algorithmic approaches: the Chambolle-Pock, ADMM and semismooth Newton algorithms. Algorithmic details and an explicit implementation is presented and the solutions are then compared numerically in a simulation study and on various test images. We thereby recommend the Chambolle-Pock algorithm in most cases for its fast convergence. We stress that our analysis can also be transferred to signal recovery and other denoising problems to recover more general objects whenever it is possible to borrow statistical strength from data patches of similar object structure.


page 5

page 15

page 16

page 19

page 20

page 21


A Variational View on Statistical Multiscale Estimation

We present a unifying view on various statistical estimation techniques ...

Nonparametric estimation of copulas and copula densities by orthogonal projections

In this paper we study nonparametric estimators of copulas and copula de...

An Online Projection Estimator for Nonparametric Regression in Reproducing Kernel Hilbert Spaces

The goal of nonparametric regression is to recover an underlying regress...

A linear time algorithm for multiscale quantile simulation

Change-point problems have appeared in a great many applications for exa...

Multiscale inference and long-run variance estimation in nonparametric regression with time series errors

In this paper, we develop new multiscale methods to test qualitative hyp...

Efficient First-Order Algorithms for Adaptive Signal Denoising

We consider the problem of discrete-time signal denoising, focusing on a...

A Multiscale Scan Statistic for Adaptive Submatrix Localization

We consider the problem of localizing a submatrix with larger-than-usual...

Please sign up or login with your details

Forgot password? Click here to reset