Convex Formulation for Kernel PCA and its Use in Semi-Supervised Learning

by   Carlos M. Alaíz, et al.

In this paper, Kernel PCA is reinterpreted as the solution to a convex optimization problem. Actually, there is a constrained convex problem for each principal component, so that the constraints guarantee that the principal component is indeed a solution, and not a mere saddle point. Although these insights do not imply any algorithmic improvement, they can be used to further understand the method, formulate possible extensions and properly address them. As an example, a new convex optimization problem for semi-supervised classification is proposed, which seems particularly well-suited whenever the number of known labels is small. Our formulation resembles a Least Squares SVM problem with a regularization parameter multiplied by a negative sign, combined with a variational principle for Kernel PCA. Our primal optimization principle for semi-supervised learning is solved in terms of the Lagrange multipliers. Numerical experiments in several classification tasks illustrate the performance of the proposed model in problems with only a few labeled data.


page 3

page 5


Learning to Impute: A General Framework for Semi-supervised Learning

Recent semi-supervised learning methods have shown to achieve comparable...

Robust PCA via Regularized REAPER with a Matrix-Free Proximal Algorithm

Principal component analysis (PCA) is known to be sensitive to outliers,...

Exact Guarantees on the Absence of Spurious Local Minima for Non-negative Robust Principal Component Analysis

This work is concerned with the non-negative robust principal component ...

Principal Component Classification

We propose to directly compute classification estimates by learning feat...

Convex Formulations for Fair Principal Component Analysis

Though there is a growing body of literature on fairness for supervised ...

Feature Selection based on Principal Component Analysis for Underwater Source Localization by Deep Learning

In this paper, we propose an interpretable feature selection method base...

Semi-supervised Fisher vector network

In this work we explore how the architecture proposed in [8], which expr...

Please sign up or login with your details

Forgot password? Click here to reset