DC approximation approaches for sparse optimization

by   Hoai An Le Thi, et al.

Sparse optimization refers to an optimization problem involving the zero-norm in objective or constraints. In this paper, nonconvex approximation approaches for sparse optimization have been studied with a unifying point of view in DC (Difference of Convex functions) programming framework. Considering a common DC approximation of the zero-norm including all standard sparse inducing penalty functions, we studied the consistency between global minimums (resp. local minimums) of approximate and original problems. We showed that, in several cases, some global minimizers (resp. local minimizers) of the approximate problem are also those of the original problem. Using exact penalty techniques in DC programming, we proved stronger results for some particular approximations, namely, the approximate problem, with suitable parameters, is equivalent to the original problem. The efficiency of several sparse inducing penalty functions have been fully analyzed. Four DCA (DC Algorithm) schemes were developed that cover all standard algorithms in nonconvex sparse approximation approaches as special versions. They can be viewed as, an ℓ _1-perturbed algorithm / reweighted-ℓ _1 algorithm / reweighted-ℓ _1 algorithm. We offer a unifying nonconvex approximation approach, with solid theoretical tools as well as efficient algorithms based on DC programming and DCA, to tackle the zero-norm and sparse optimization. As an application, we implemented our methods for the feature selection in SVM (Support Vector Machine) problem and performed empirical comparative numerical experiments on the proposed algorithms with various approximation functions.


page 1

page 2

page 3

page 4


A Unifying Framework of High-Dimensional Sparse Estimation with Difference-of-Convex (DC) Regularizations

Under the linear regression framework, we study the variable selection p...

Approximate Denial Constraints

The problem of mining integrity constraints from data has been extensive...

Difference of Submodular Minimization via DC Programming

Minimizing the difference of two submodular (DS) functions is a problem ...

Coordinate Descent Methods for DC Minimization

Difference-of-Convex (DC) minimization, referring to the problem of mini...

DAG-Inducing Problems and Algorithms

In this paper, we show that in a parallel processing system, if a direct...

M-estimation with the Trimmed l1 Penalty

We study high-dimensional M-estimators with the trimmed ℓ_1 penalty. Whi...

Equivalent Lipschitz surrogates for zero-norm and rank optimization problems

This paper proposes a mechanism to produce equivalent Lipschitz surrogat...

Please sign up or login with your details

Forgot password? Click here to reset