Forgettable Federated Linear Learning with Certified Data Removal

by   Ruinan Jin, et al.

Federated learning (FL) is a trending distributed learning framework that enables collaborative model training without data sharing. Machine learning models trained on datasets can potentially expose the private information of the training data, revealing details about individual data records. In this study, we focus on the FL paradigm that grants clients the “right to be forgotten”. The forgettable FL framework should bleach its global model weights as it has never seen that client and hence does not reveal any information about the client. To this end, we propose the Forgettable Federated Linear Learning (2F2L) framework featured with novel training and data removal strategies. The training pipeline, named Federated linear training, employs linear approximation on the model parameter space to enable our 2F2L framework work for deep neural networks while achieving comparable results with canonical neural network training. We also introduce FedRemoval, an efficient and effective removal strategy that tackles the computational challenges in FL by approximating the Hessian matrix using public server data from the pretrained model. Unlike the previous uncertified and heuristic machine unlearning methods in FL, we provide theoretical guarantees by bounding the differences of model weights by our FedRemoval and that from retraining from scratch. Experimental results on MNIST and Fashion-MNIST datasets demonstrate the effectiveness of our method in achieving a balance between model accuracy and information removal, outperforming baseline strategies and approaching retraining from scratch.


PFL-MoE: Personalized Federated Learning Based on Mixture of Experts

Federated learning (FL) is an emerging distributed machine learning para...

Sequential Informed Federated Unlearning: Efficient and Provable Client Unlearning in Federated Optimization

The aim of Machine Unlearning (MU) is to provide theoretical guarantees ...

Federated Unlearning: How to Efficiently Erase a Client in FL?

With privacy legislation empowering users with the right to be forgotten...

Adaptive Federated Learning with Auto-Tuned Clients

Federated learning (FL) is a distributed machine learning framework wher...

The Right to be Forgotten in Federated Learning: An Efficient Realization with Rapid Retraining

In Machine Learning, the emergence of the right to be forgotten gave bir...

DACFL: Dynamic Average Consensus Based Federated Learning in Decentralized Topology

Federated learning (FL) is a burgeoning distributed machine learning fra...

DeepObliviate: A Powerful Charm for Erasing Data Residual Memory in Deep Neural Networks

Machine unlearning has great significance in guaranteeing model security...

Please sign up or login with your details

Forgot password? Click here to reset