HPN: Personalized Federated Hyperparameter Optimization

by   Anda Cheng, et al.

Numerous research studies in the field of federated learning (FL) have attempted to use personalization to address the heterogeneity among clients, one of FL's most crucial and challenging problems. However, existing works predominantly focus on tailoring models. Yet, due to the heterogeneity of clients, they may each require different choices of hyperparameters, which have not been studied so far. We pinpoint two challenges of personalized federated hyperparameter optimization (pFedHPO): handling the exponentially increased search space and characterizing each client without compromising its data privacy. To overcome them, we propose learning a HyperParameter Network (HPN) fed with client encoding to decide personalized hyperparameters. The client encoding is calculated with a random projection-based procedure to protect each client's privacy. Besides, we design a novel mechanism to debias the low-fidelity function evaluation samples for learning HPN. We conduct extensive experiments on FL tasks from various domains, demonstrating the superiority of HPN.


page 4

page 5


PI-FL: Personalized and Incentivized Federated Learning

Personalized FL has been widely used to cater to heterogeneity challenge...

Efficient Model Personalization in Federated Learning via Client-Specific Prompt Generation

Federated learning (FL) emerges as a decentralized learning framework wh...

Evaluation of Hyperparameter-Optimization Approaches in an Industrial Federated Learning System

Federated Learning (FL) decouples model training from the need for direc...

On Noisy Evaluation in Federated Hyperparameter Tuning

Hyperparameter tuning is critical to the success of federated learning a...

Federated Transfer-Ordered-Personalized Learning for Driver Monitoring Application

Federated learning (FL) shines through in the internet of things (IoT) w...

FedPop: Federated Population-based Hyperparameter Tuning

Federated Learning (FL) is a distributed machine learning (ML) paradigm,...

Personalized Subgraph Federated Learning

In real-world scenarios, subgraphs of a larger global graph may be distr...

Please sign up or login with your details

Forgot password? Click here to reset