Low-Rank Generalized Linear Bandit Problems

by   Yangyi Lu, et al.

In a low-rank linear bandit problem, the reward of an action (represented by a matrix of size d_1 × d_2) is the inner product between the action and an unknown low-rank matrix Θ^*. We propose an algorithm based on a novel combination of online-to-confidence-set conversion <cit.> and the exponentially weighted average forecaster constructed by a covering of low-rank matrices. In T rounds, our algorithm achieves O((d_1+d_2)^3/2√(rT)) regret that improves upon the standard linear bandit regret bound of O(d_1d_2√(T)) when the rank of Θ^*: r ≪min{d_1,d_2}. We also extend our algorithmic approach to the generalized linear setting to get an algorithm which enjoys a similar bound under regularity conditions on the link function. To get around the computational intractability of covering based approaches, we propose an efficient algorithm by extending the "Explore-Subspace-Then-Refine" algorithm of <cit.>. Our efficient algorithm achieves O((d_1+d_2)^3/2√(rT)) regret under a mild condition on the action set X and the r-th singular value of Θ^*. Our upper bounds match the conjectured lower bound of <cit.> for a subclass of low-rank linear bandit problems. Further, we show that existing lower bounds for the sparse linear bandit problem strongly suggest that our regret bounds are unimprovable. To complement our theoretical contributions, we also conduct experiments to demonstrate that our algorithm can greatly outperform the performance of the standard linear bandit approach when Θ^* is low-rank.


page 1

page 2

page 3

page 4


A Simple Unified Framework for High Dimensional Bandit Problems

Stochastic high dimensional bandit problems with low dimensional structu...

Bilinear Bandits with Low-rank Structure

We introduce the bilinear bandit problem with low-rank structure where a...

Stochastic continuum armed bandit problem of few linear parameters in high dimensions

We consider a stochastic continuum armed bandit problem where the arms a...

Episodic Linear Quadratic Regulators with Low-rank Transitions

Linear Quadratic Regulators (LQR) achieve enormous successful real-world...

Bandit Multiclass Linear Classification: Efficient Algorithms for the Separable Case

We study the problem of efficient online multiclass linear classificatio...

UniRank: Unimodal Bandit Algorithm for Online Ranking

We tackle a new emerging problem, which is finding an optimal monopartit...

Unimodal Mono-Partite Matching in a Bandit Setting

We tackle a new emerging problem, which is finding an optimal monopartit...

Please sign up or login with your details

Forgot password? Click here to reset