Exploring the Benefits of Visual Prompting in Differential Privacy

by   Yizhe Li, et al.

Visual Prompting (VP) is an emerging and powerful technique that allows sample-efficient adaptation to downstream tasks by engineering a well-trained frozen source model. In this work, we explore the benefits of VP in constructing compelling neural network classifiers with differential privacy (DP). We explore and integrate VP into canonical DP training methods and demonstrate its simplicity and efficiency. In particular, we discover that VP in tandem with PATE, a state-of-the-art DP training method that leverages the knowledge transfer from an ensemble of teachers, achieves the state-of-the-art privacy-utility trade-off with minimum expenditure of privacy budget. Moreover, we conduct additional experiments on cross-domain image classification with a sufficient domain gap to further unveil the advantage of VP in DP. Lastly, we also conduct extensive ablation studies to validate the effectiveness and contribution of VP under DP consideration.


Differential Privacy Has Disparate Impact on Model Accuracy

Differential privacy (DP) is a popular mechanism for training machine le...

Accuracy, Interpretability, and Differential Privacy via Explainable Boosting

We show that adding differential privacy to Explainable Boosting Machine...

Improving Deep Learning with Differential Privacy using Gradient Encoding and Denoising

Deep learning models leak significant amounts of information about their...

Learning to Generate Image Embeddings with User-level Differential Privacy

Small on-device models have been successfully trained with user-level di...

DPNAS: Neural Architecture Search for Deep Learning with Differential Privacy

Training deep neural networks (DNNs) for meaningful differential privacy...

DeePoint: Pointing Recognition and Direction Estimation From A Fixed View

In this paper, we realize automatic visual recognition and direction est...

The Impact of De-Identification on Single-Year-of-Age Counts in the U.S. Census

In 2020, the U.S. Census Bureau transitioned from data swapping to diffe...

Please sign up or login with your details

Forgot password? Click here to reset