DeepAI AI Chat
Log In Sign Up

Make Landscape Flatter in Differentially Private Federated Learning

by   Yifan Shi, et al.
Nanjing University
Tsinghua University

To defend the inference attacks and mitigate the sensitive information leakages in Federated Learning (FL), client-level Differentially Private FL (DPFL) is the de-facto standard for privacy protection by clipping local updates and adding random noise. However, existing DPFL methods tend to make a sharper loss landscape and have poorer weight perturbation robustness, resulting in severe performance degradation. To alleviate these issues, we propose a novel DPFL algorithm named DP-FedSAM, which leverages gradient perturbation to mitigate the negative impact of DP. Specifically, DP-FedSAM integrates Sharpness Aware Minimization (SAM) optimizer to generate local flatness models with better stability and weight perturbation robustness, which results in the small norm of local updates and robustness to DP noise, thereby improving the performance. From the theoretical perspective, we analyze in detail how DP-FedSAM mitigates the performance degradation induced by DP. Meanwhile, we give rigorous privacy guarantees with Rényi DP and present the sensitivity analysis of local updates. At last, we empirically confirm that our algorithm achieves state-of-the-art (SOTA) performance compared with existing SOTA baselines in DPFL.


page 2

page 13


Towards the Flatter Landscape and Better Generalization in Federated Learning under Client-level Differential Privacy

To defend the inference attacks and mitigate the sensitive information l...

Federated Learning with Sparsified Model Perturbation: Improving Accuracy under Client-Level Differential Privacy

Federated learning (FL) that enables distributed clients to collaborativ...

Differentially Private Federated Bayesian Optimization with Distributed Exploration

Bayesian optimization (BO) has recently been extended to the federated l...

DPVIm: Differentially Private Variational Inference Improved

Differentially private (DP) release of multidimensional statistics typic...

Hierarchical Federated Learning with Privacy

Federated learning (FL), where data remains at the federated clients, an...

Practical Differentially Private and Byzantine-resilient Federated Learning

Privacy and Byzantine resilience are two indispensable requirements for ...

One-shot Empirical Privacy Estimation for Federated Learning

Privacy auditing techniques for differentially private (DP) algorithms a...