DeepAI AI Chat
Log In Sign Up

Separability and Scatteredness (S S) Ratio-Based Efficient SVM Regularization Parameter, Kernel, and Kernel Parameter Selection

by   Mahdi Shamsi, et al.

Support Vector Machine (SVM) is a robust machine learning algorithm with broad applications in classification, regression, and outlier detection. SVM requires tuning the regularization parameter (RP) which controls the model capacity and the generalization performance. Conventionally, the optimum RP is found by comparison of a range of values through the Cross-Validation (CV) procedure. In addition, for non-linearly separable data, the SVM uses kernels where a set of kernels, each with a set of parameters, denoted as a grid of kernels, are considered. The optimal choice of RP and the grid of kernels is through the grid-search of CV. By stochastically analyzing the behavior of the regularization parameter, this work shows that the SVM performance can be modeled as a function of separability and scatteredness (S S) of the data. Separability is a measure of the distance between classes, and scatteredness is the ratio of the spread of data points. In particular, for the hinge loss cost function, an S S ratio-based table provides the optimum RP. The S S ratio is a powerful value that can automatically detect linear or non-linear separability before using the SVM algorithm. The provided S S ratio-based table can also provide the optimum kernel and its parameters before using the SVM algorithm. Consequently, the computational complexity of the CV grid-search is reduced to only one time use of the SVM. The simulation results on the real dataset confirm the superiority and efficiency of the proposed approach in the sense of computational complexity over the grid-search CV method.


page 1

page 2

page 3

page 4


Geometric Insights into Support Vector Machine Behavior using the KKT Conditions

The Support Vector Machine (SVM) is a powerful and widely used classific...

Heuristical choice of SVM parameters

Support Vector Machine (SVM) is one of the most popular classification m...

Faster SVM Training via Conjugate SMO

We propose an improved version of the SMO algorithm for training classif...

Practical Selection of SVM Supervised Parameters with Different Feature Representations for Vowel Recognition

It is known that the classification performance of Support Vector Machin...

Fast model selection by limiting SVM training times

Kernelized Support Vector Machines (SVMs) are among the best performing ...

Engineering multilevel support vector machines

The computational complexity of solving nonlinear support vector machine...

TrIK-SVM : an alternative decomposition for kernel methods in Krein spaces

The proposed work aims at proposing a alternative kernel decomposition i...