Take Your Pick: Enabling Effective Personalized Federated Learning within Low-dimensional Feature Space

by   Guogang Zhu, et al.

Personalized federated learning (PFL) is a popular framework that allows clients to have different models to address application scenarios where clients' data are in different domains. The typical model of a client in PFL features a global encoder trained by all clients to extract universal features from the raw data and personalized layers (e.g., a classifier) trained using the client's local data. Nonetheless, due to the differences between the data distributions of different clients (aka, domain gaps), the universal features produced by the global encoder largely encompass numerous components irrelevant to a certain client's local task. Some recent PFL methods address the above problem by personalizing specific parameters within the encoder. However, these methods encounter substantial challenges attributed to the high dimensionality and non-linearity of neural network parameter space. In contrast, the feature space exhibits a lower dimensionality, providing greater intuitiveness and interpretability as compared to the parameter space. To this end, we propose a novel PFL framework named FedPick. FedPick achieves PFL in the low-dimensional feature space by selecting task-relevant features adaptively for each client from the features generated by the global encoder based on its local data distribution. It presents a more accessible and interpretable implementation of PFL compared to those methods working in the parameter space. Extensive experimental results show that FedPick could effectively select task-relevant features for each client and improve model performance in cross-domain FL.


page 1

page 11


Personalized Federated Learning with First Order Model Optimization

While federated learning traditionally aims to train a single global mod...

Cross-domain Federated Object Detection

Detection models trained by one party (server) may face severe performan...

Personalised Federated Learning On Heterogeneous Feature Spaces

Most personalised federated learning (FL) approaches assume that raw dat...

Intent Detection at Scale: Tuning a Generic Model using Relevant Intents

Accurately predicting the intent of customer support requests is vital f...

Far from Asymptopia

Inference from limited data requires a notion of measure on parameter sp...

Bold but Cautious: Unlocking the Potential of Personalized Federated Learning through Cautiously Aggressive Collaboration

Personalized federated learning (PFL) reduces the impact of non-independ...

Federated Geometric Monte Carlo Clustering to Counter Non-IID Datasets

Federated learning allows clients to collaboratively train models on dat...

Please sign up or login with your details

Forgot password? Click here to reset