An Introduction to Bi-level Optimization: Foundations and Applications in Signal Processing and Machine Learning

by   Yihua Zhang, et al.

Recently, bi-level optimization (BLO) has taken center stage in some very exciting developments in the area of signal processing (SP) and machine learning (ML). Roughly speaking, BLO is a classical optimization problem that involves two levels of hierarchy (i.e., upper and lower levels), wherein obtaining the solution to the upper-level problem requires solving the lower-level one. BLO has become popular largely because it is powerful in modeling problems in SP and ML, among others, that involve optimizing nested objective functions. Prominent applications of BLO range from resource allocation for wireless systems to adversarial machine learning. In this work, we focus on a class of tractable BLO problems that often appear in SP and ML applications. We provide an overview of some basic concepts of this class of BLO problems, such as their optimality conditions, standard algorithms (including their optimization principles and practical implementations), as well as how they can be leveraged to obtain state-of-the-art results for a number of key SP and ML applications. Further, we discuss some recent advances in BLO theory, its implications for applications, and point out some limitations of the state-of-the-art that require significant future research efforts. Overall, we hope that this article can serve to accelerate the adoption of BLO as a generic tool to model, analyze, and innovate on a wide array of emerging SP and ML applications.


page 3

page 5

page 6

page 12


Machine Learning Tips and Tricks for Power Line Communications

A great deal of attention has been recently given to Machine Learning (M...

Boosting Combinatorial Problem Modeling with Machine Learning

In the past few years, the area of Machine Learning (ML) has witnessed t...

An Overview of Machine Learning Approaches in Wireless Mesh Networks

Wireless Mesh Networks (WMNs) have been extensively studied for nearly t...

Machine Learning in Nano-Scale Biomedical Engineering

Machine learning (ML) empowers biomedical systems with the capability to...

Efficient and Direct Inference of Heart Rate Variability using Both Signal Processing and Machine Learning

Heart Rate Variability (HRV) measures the variation of the time between ...

Roadmap on Signal Processing for Next Generation Measurement Systems

Signal processing is a fundamental component of almost any sensor-enable...

When Lempel-Ziv-Welch Meets Machine Learning: A Case Study of Accelerating Machine Learning using Coding

In this paper we study the use of coding techniques to accelerate machin...

Please sign up or login with your details

Forgot password? Click here to reset