Fully Privacy-Preserving Federated Representation Learning via Secure Embedding Aggregation

by   Jiaxiang Tang, et al.

We consider a federated representation learning framework, where with the assistance of a central server, a group of N distributed clients train collaboratively over their private data, for the representations (or embeddings) of a set of entities (e.g., users in a social network). Under this framework, for the key step of aggregating local embeddings trained at the clients in a private manner, we develop a secure embedding aggregation protocol named SecEA, which provides information-theoretical privacy guarantees for the set of entities and the corresponding embeddings at each client simultaneously, against a curious server and up to T < N/2 colluding clients. As the first step of SecEA, the federated learning system performs a private entity union, for each client to learn all the entities in the system without knowing which entities belong to which clients. In each aggregation round, the local embeddings are secretly shared among the clients using Lagrange interpolation, and then each client constructs coded queries to retrieve the aggregated embeddings for the intended entities. We perform comprehensive experiments on various representation learning tasks to evaluate the utility and efficiency of SecEA, and empirically demonstrate that compared with embedding aggregation protocols without (or with weaker) privacy guarantees, SecEA incurs negligible performance loss (within 5 additional computation latency of SecEA diminishes for training deeper models on larger datasets.


FastSecAgg: Scalable Secure Aggregation for Privacy-Preserving Federated Learning

Recent attacks on federated learning demonstrate that keeping the traini...

Secure Federated Submodel Learning

Federated learning was proposed with an intriguing vision of achieving c...

Efficient Secure Aggregation for Privacy-Preserving Federated Machine Learning

Federated learning introduces a novel approach to training machine learn...

Efficient Federated Learning on Knowledge Graphs via Privacy-preserving Relation Embedding Aggregation

Federated Learning (FL) on knowledge graphs (KGs) has yet to be as well ...

Private Federated Submodel Learning via Private Set Union

We consider the federated submodel learning (FSL) problem and propose an...

Robust Decentralized Learning for Neural Networks

In decentralized learning, data is distributed among local clients which...

Tackling Dynamics in Federated Incremental Learning with Variational Embedding Rehearsal

Federated Learning is a fast growing area of ML where the training datas...

Please sign up or login with your details

Forgot password? Click here to reset