Risk bounds when learning infinitely many response functions by ordinary linear regression

06/16/2020
by   Vincent Plassier, et al.
0

Consider the problem of learning a large number of response functions simultaneously based on the same input variables. The training data consist of a single independent random sample of the input variables drawn from a common distribution together with the associated responses. The input variables are mapped into a high-dimensional linear space, called the feature space, and the response functions are modelled as linear functionals of the mapped features, with coefficients calibrated via ordinary least squares. We provide convergence guarantees on the worst-case excess prediction risk by controlling the convergence rate of the excess risk uniformly in the response function. The dimension of the feature map is allowed to tend to infinity with the sample size. The collection of response functions, although potentiallyinfinite, is supposed to have a finite Vapnik-Chervonenkis dimension. The bound derived can be applied when building multiple surrogate models in a reasonable computing time.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
08/14/2020

Provable More Data Hurt in High Dimensional Least Squares Estimator

This paper investigates the finite-sample prediction risk of the high-di...
research
07/05/2016

Risk Bounds for High-dimensional Ridge Function Combinations Including Neural Networks

Let f^ be a function on R^d satisfying a spectral norm condition. Fo...
research
02/18/2021

Adjusting the Benjamini-Hochberg method for controlling the false discovery rate in knockoff assisted variable selection

This paper revisits the knockoff-based multiple testing setup considered...
research
01/03/2016

Dimensionality-Dependent Generalization Bounds for k-Dimensional Coding Schemes

The k-dimensional coding schemes refer to a collection of methods that a...
research
07/07/2023

Scalable High-Dimensional Multivariate Linear Regression for Feature-Distributed Data

Feature-distributed data, referred to data partitioned by features and s...
research
09/05/2018

Conditional predictive inference for high-dimensional stable algorithms

We investigate generically applicable and intuitively appealing predicti...
research
06/05/2020

Expressivity of expand-and-sparsify representations

A simple sparse coding mechanism appears in the sensory systems of sever...

Please sign up or login with your details

Forgot password? Click here to reset