Private Online Prediction from Experts: Separations and Faster Rates

10/24/2022
by   Hilal Asi, et al.
0

Online prediction from experts is a fundamental problem in machine learning and several works have studied this problem under privacy constraints. We propose and analyze new algorithms for this problem that improve over the regret bounds of the best existing algorithms for non-adaptive adversaries. For approximate differential privacy, our algorithms achieve regret bounds of Õ(√(T log d) + log d/ε) for the stochastic setting and Õ(√(T log d) + T^1/3log d/ε) for oblivious adversaries (where d is the number of experts). For pure DP, our algorithms are the first to obtain sub-linear regret for oblivious adversaries in the high-dimensional regime d ≥ T. Moreover, we prove new lower bounds for adaptive adversaries. Our results imply that unlike the non-private setting, there is a strong separation between the optimal regret for adaptive and non-adaptive adversaries for this problem. Our lower bounds also show a separation between pure and approximate differential privacy for adaptive adversaries where the latter is necessary to achieve the non-private O(√(T)) regret.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
02/27/2023

Near-Optimal Algorithms for Private Online Optimization in the Realizable Regime

We consider online learning problems in the realizable setting, where th...
research
03/29/2023

Non-Asymptotic Lower Bounds For Training Data Reconstruction

We investigate semantic guarantees of private learning algorithms for th...
research
05/24/2021

Cascading Bandit under Differential Privacy

This paper studies differential privacy (DP) and local differential priv...
research
02/21/2020

Privately Learning Markov Random Fields

We consider the problem of learning Markov Random Fields (including the ...
research
10/24/2020

Differentially Private Online Submodular Maximization

In this work we consider the problem of online submodular maximization u...
research
06/25/2020

Active Online Domain Adaptation

Online machine learning systems need to adapt to domain shifts. Meanwhil...
research
02/18/2013

Online Learning with Switching Costs and Other Adaptive Adversaries

We study the power of different types of adaptive (nonoblivious) adversa...

Please sign up or login with your details

Forgot password? Click here to reset