Closure Properties for Private Classification and Online Prediction

03/10/2020
by   Noga Alon, et al.
0

Let H be a class of boolean functions and consider acomposed class H' that is derived from H using some arbitrary aggregation rule (for example, H' may be the class of all 3-wise majority votes of functions in H). We upper bound the Littlestone dimension of H' in terms of that of H. The bounds are proved using combinatorial arguments that exploit a connection between the Littlestone dimension and Thresholds. As a corollary, we derive closure properties for online learning and private PAC learning. The derived bounds on the Littlestone dimension exhibit an undesirable super-exponential dependence. For private learning, we prove close to optimal bounds that circumvents this suboptimal dependency. The improved bounds on the sample complexity of private learning are derived algorithmically via transforming a private learner for the original class H to a private learner for the composed class H'. Using the same ideas we show that any (proper or improper) private algorithm that learns a class of functions H in the realizable case (i.e., when the examples are labeled by some function in the class) can be transformed to a private algorithm that learns the class H in the agnostic case.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
08/11/2023

Private Distribution Learning with Public Data: The View from Sample Compression

We study the problem of private distribution learning with access to pub...
research
07/22/2021

Multiclass versus Binary Differentially Private PAC Learning

We show a generic reduction from multiclass differentially private PAC l...
research
07/31/2019

Privately Answering Classification Queries in the Agnostic PAC Model

We revisit the problem of differentially private release of classificati...
research
04/23/2020

Private Query Release Assisted by Public Data

We study the problem of differentially private query release assisted by...
research
03/14/2018

Model-Agnostic Private Learning via Stability

We design differentially private learning algorithms that are agnostic t...
research
01/04/2023

Online Learning of Smooth Functions

In this paper, we study the online learning of real-valued functions whe...
research
07/07/2020

Near-tight closure bounds for Littlestone and threshold dimensions

We study closure properties for the Littlestone and threshold dimensions...

Please sign up or login with your details

Forgot password? Click here to reset