A sufficient condition on monotonic increase of the number of nonzero entry in the optimizer of L1 norm penalized least-square problem

by   J. Duan, et al.

The ℓ-1 norm based optimization is widely used in signal processing, especially in recent compressed sensing theory. This paper studies the solution path of the ℓ-1 norm penalized least-square problem, whose constrained form is known as Least Absolute Shrinkage and Selection Operator (LASSO). A solution path is the set of all the optimizers with respect to the evolution of the hyperparameter (Lagrange multiplier). The study of the solution path is of great significance in viewing and understanding the profile of the tradeoff between the approximation and regularization terms. If the solution path of a given problem is known, it can help us to find the optimal hyperparameter under a given criterion such as the Akaike Information Criterion. In this paper we present a sufficient condition on ℓ-1 norm penalized least-square problem. Under this sufficient condition, the number of nonzero entries in the optimizer or solution vector increases monotonically when the hyperparameter decreases. We also generalize the result to the often used total variation case, where the ℓ-1 norm is taken over the first order derivative of the solution vector. We prove that the proposed condition has intrinsic connections with the condition given by Donoho, et al Donoho08 and the positive cone condition by Efron el alEfron04. However, the proposed condition does not need to assume the sparsity level of the signal as required by Donoho et al's condition, and is easier to verify than Efron, et al's positive cone condition when being used for practical applications.


The Geometry of Uniqueness and Model Selection of Penalized Estimators including SLOPE, LASSO, and Basis Pursuit

We provide a necessary and sufficient condition for the uniqueness of pe...

On the connections between algorithmic regularization and penalization for convex losses

In this work we establish the equivalence of algorithmic regularization ...

The tangential cone condition for some coefficient identification model problems in parabolic PDEs

The tangential condition was introduced in [Hanke et al., 95] as a suffi...

Pattern Recovery in Penalized and Thresholded Estimation and its Geometry

We consider the framework of penalized estimation where the penalty term...

Edge convex smooth interpolation curve networks with minimum L_∞-norm of the second derivative

We consider the extremal problem of interpolation of convex scattered da...

The Proximal Operator of the Piece-wise Exponential Function and Its Application in Compressed Sensing

This paper characterizes the proximal operator of the piece-wise exponen...

Wireless Energy Transfer to a Pair of Energy Receivers using Signal Strength Feedback

This paper focuses on wireless energy transfer (WET) to a pair of low co...

Please sign up or login with your details

Forgot password? Click here to reset