Geometric Insights into Support Vector Machine Behavior using the KKT Conditions

by   Iain Carmichael, et al.
University of North Carolina at Chapel Hill

The Support Vector Machine (SVM) is a powerful and widely used classification algorithm. Its performance is well known to be impacted by a tuning parameter which is frequently selected by cross-validation. This paper uses the Karush-Kuhn-Tucker conditions to provide rigorous mathematical proof for new insights into the behavior of SVM in the large and small tuning parameter regimes. These insights provide perhaps unexpected relationships between SVM and naive Bayes and maximal data piling directions. We explore how characteristics of the training data affect the behavior of SVM in many cases including: balanced vs. unbalanced classes, low vs. high dimension, separable vs. non-separable data. These results present a simple explanation of SVM's behavior as a function of the tuning parameter. We also elaborate on the geometry of complete data piling directions in high dimensional space. The results proved in this paper suggest important implications for tuning SVM with cross-validation.


page 1

page 2

page 3

page 4


Comparing SVM and Naive Bayes classifiers for text categorization with Wikitology as knowledge enrichment

The activity of labeling of documents according to their content is know...

Optimal Model Averaging of Support Vector Machines in Diverging Model Spaces

Support vector machine (SVM) is a powerful classification method that ha...

Heuristical choice of SVM parameters

Support Vector Machine (SVM) is one of the most popular classification m...

Optimised one-class classification performance

We provide a thorough treatment of hyperparameter optimisation for three...

Exact high-dimensional asymptotics for support vector machine

Support vector machine (SVM) is one of the most widely used classificati...

An alternative to SVM Method for Data Classification

Support vector machine (SVM), is a popular kernel method for data classi...

Please sign up or login with your details

Forgot password? Click here to reset