A Precise Performance Analysis of Support Vector Regression

by   Houssem Sifaou, et al.

In this paper, we study the hard and soft support vector regression techniques applied to a set of n linear measurements of the form y_i=β_⋆^T x_i +n_i where β_⋆ is an unknown vector, { x_i}_i=1^n are the feature vectors and {n_i}_i=1^n model the noise. Particularly, under some plausible assumptions on the statistical distribution of the data, we characterize the feasibility condition for the hard support vector regression in the regime of high dimensions and, when feasible, derive an asymptotic approximation for its risk. Similarly, we study the test risk for the soft support vector regression as a function of its parameters. Our results are then used to optimally tune the parameters intervening in the design of hard and soft support vector regression algorithms. Based on our analysis, we illustrate that adding more samples may be harmful to the test performance of support vector regression, while it is always beneficial when the parameters are optimally selected. Such a result reminds a similar phenomenon observed in modern learning architectures according to which optimally tuned architectures present a decreasing test performance curve with respect to the number of samples.


page 1

page 2

page 3

page 4


On the Precise Error Analysis of Support Vector Machines

This paper investigates the asymptotic behavior of the soft-margin and h...

A working likelihood approach to support vector regression with a data-driven insensitivity parameter

The insensitive parameter in support vector regression determines the se...

Maximal Margin Distribution Support Vector Regression with coupled Constraints-based Convex Optimization

Support vector regression (SVR) is one of the most popular machine learn...

Linear Support Vector Regression with Linear Constraints

This paper studies the addition of linear constraints to the Support Vec...

Predicting the Critical Number of Layers for Hierarchical Support Vector Regression

Hierarchical support vector regression (HSVR) models a function from dat...

Support Vector Regression: Risk Quadrangle Framework

This paper investigates Support Vector Regression (SVR) in the context o...

High-low level support vector regression prediction approach (HL-SVR) for data modeling with input parameters of unequal sample sizes

Support vector regression (SVR) has been widely used to reduce the high ...

Please sign up or login with your details

Forgot password? Click here to reset