No Cross-Validation Required: An Analytical Framework for Regularized Mixed-Integer Problems (Extended Version)

08/04/2020
by   Behrad Soleimani, et al.
0

This paper develops a method to obtain the optimal value for the regularization coefficient in a general mixed-integer problem (MIP). This approach eliminates the cross-validation performed in the existing penalty techniques to obtain a proper value for the regularization coefficient. We obtain this goal by proposing an alternating method to solve MIPs. First, via regularization, we convert the MIP into a more mathematically tractable form. Then, we develop an iterative algorithm to update the solution along with the regularization (penalty) coefficient. We show that our update procedure guarantees the convergence of the algorithm. Moreover, assuming the objective function is continuously differentiable, we derive the convergence rate, a lower bound on the value of regularization coefficient, and an upper bound on the number of iterations required for the convergence. We use a radio access technology (RAT) selection problem in a heterogeneous network to benchmark the performance of our method. Simulation results demonstrate near-optimality of the solution and consistency of the convergence behavior with obtained theoretical bounds.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
02/09/2015

Regularization Path of Cross-Validation Error Lower Bounds

Careful tuning of a regularization parameter is indispensable in many ma...
research
05/29/2020

Consistent Second-Order Conic Integer Programming for Learning Bayesian Networks

Bayesian Networks (BNs) represent conditional probability relations amon...
research
02/05/2019

Consistent Risk Estimation in High-Dimensional Linear Regression

Risk estimation is at the core of many learning systems. The importance ...
research
08/07/2018

Efficient and Effective L_0 Feature Selection

Because of continuous advances in mathematical programing, Mix Integer O...
research
07/15/2019

Improved Penalty Algorithm for Mixed Integer PDE Constrained Optimization (MIPDECO) Problems

Optimal control problems including partial differential equation (PDE) a...
research
12/30/2018

On Cross-validation for Sparse Reduced Rank Regression

In high-dimensional data analysis, regularization methods pursuing spars...
research
08/25/2015

OCReP: An Optimally Conditioned Regularization for Pseudoinversion Based Neural Training

In this paper we consider the training of single hidden layer neural net...

Please sign up or login with your details

Forgot password? Click here to reset