Stochastic Contextual Bandits with Graph-based Contexts

by   Jittat Fakcharoenphol, et al.

We naturally generalize the on-line graph prediction problem to a version of stochastic contextual bandit problems where contexts are vertices in a graph and the structure of the graph provides information on the similarity of contexts. More specifically, we are given a graph G=(V,E), whose vertex set V represents contexts with unknown vertex label y. In our stochastic contextual bandit setting, vertices with the same label share the same reward distribution. The standard notion of instance difficulties in graph label prediction is the cutsize f defined to be the number of edges whose end points having different labels. For line graphs and trees we present an algorithm with regret bound of Õ(T^2/3K^1/3f^1/3) where K is the number of arms. Our algorithm relies on the optimal stochastic bandit algorithm by Zimmert and Seldin [AISTAT'19, JMLR'21]. When the best arm outperforms the other arms, the regret improves to Õ(√(KT· f)). The regret bound in the later case is comparable to other optimal contextual bandit results in more general cases, but our algorithm is easy to analyze, runs very efficiently, and does not require an i.i.d. assumption on the input context sequence. The algorithm also works with general graphs using a standard random spanning tree reduction.


page 1

page 2

page 3

page 4


Contexts can be Cheap: Solving Stochastic Contextual Bandits with Linear Bandit Algorithms

In this paper, we address the stochastic contextual linear bandit proble...

Breaking the √(T) Barrier: Instance-Independent Logarithmic Regret in Stochastic Contextual Linear Bandits

We prove an instance independent (poly) logarithmic regret for stochasti...

Bypassing the Monster: A Faster and Simpler Optimal Algorithm for Contextual Bandits under Realizability

We consider the general (stochastic) contextual bandit problem under the...

Distributed Stochastic Bandit Learning with Context Distributions

We study the problem of distributed stochastic multi-arm contextual band...

Semi-Parametric Contextual Bandits with Graph-Laplacian Regularization

Non-stationarity is ubiquitous in human behavior and addressing it in th...

Federated Stochastic Bandit Learning with Unobserved Context

We study the problem of federated stochastic multi-arm contextual bandit...

A Dimension-free Algorithm for Contextual Continuum-armed Bandits

In contextual continuum-armed bandits, the contexts x and the arms y are...

Please sign up or login with your details

Forgot password? Click here to reset