On Privacy and Personalization in Cross-Silo Federated Learning

by   Ziyu Liu, et al.

While the application of differential privacy (DP) has been well-studied in cross-device federated learning (FL), there is a lack of work considering DP for cross-silo FL, a setting characterized by a limited number of clients each containing many data subjects. In cross-silo FL, usual notions of client-level privacy are less suitable as real-world privacy regulations typically concern in-silo data subjects rather than the silos themselves. In this work, we instead consider the more realistic notion of silo-specific item-level privacy, where silos set their own privacy targets for their local examples. Under this setting, we reconsider the roles of personalization in federated learning. In particular, we show that mean-regularized multi-task learning (MR-MTL), a simple personalization framework, is a strong baseline for cross-silo FL: under stronger privacy, silos are further incentivized to "federate" with each other to mitigate DP noise, resulting in consistent improvements relative to standard baseline methods. We provide a thorough empirical study of competing methods as well as a theoretical characterization of MR-MTL for a mean estimation problem, highlighting the interplay between privacy and cross-silo data heterogeneity. Our work serves to establish baselines for private cross-silo FL as well as identify key directions of future work in this area.


ULDP-FL: Federated Learning with Across Silo User-Level Differential Privacy

Differentially Private Federated Learning (DP-FL) has garnered attention...

Understanding Clipping for Federated Learning: Convergence and Client-Level Differential Privacy

Providing privacy protection has been one of the primary motivations of ...

Adap DP-FL: Differentially Private Federated Learning with Adaptive Noise

Federated learning seeks to address the issue of isolated data islands b...

CANIFE: Crafting Canaries for Empirical Privacy Measurement in Federated Learning

Federated Learning (FL) is a setting for training machine learning model...

Private Cross-Silo Federated Learning for Extracting Vaccine Adverse Event Mentions

Federated Learning (FL) is quickly becoming a goto distributed training ...

(Amplified) Banded Matrix Factorization: A unified approach to private training

Matrix factorization (MF) mechanisms for differential privacy (DP) have ...

Towards the Flatter Landscape and Better Generalization in Federated Learning under Client-level Differential Privacy

To defend the inference attacks and mitigate the sensitive information l...

Please sign up or login with your details

Forgot password? Click here to reset