FedClassAvg: Local Representation Learning for Personalized Federated Learning on Heterogeneous Neural Networks

by   Jaehee Jang, et al.

Personalized federated learning is aimed at allowing numerous clients to train personalized models while participating in collaborative training in a communication-efficient manner without exchanging private data. However, many personalized federated learning algorithms assume that clients have the same neural network architecture, and those for heterogeneous models remain understudied. In this study, we propose a novel personalized federated learning method called federated classifier averaging (FedClassAvg). Deep neural networks for supervised learning tasks consist of feature extractor and classifier layers. FedClassAvg aggregates classifier weights as an agreement on decision boundaries on feature spaces so that clients with not independently and identically distributed (non-iid) data can learn about scarce labels. In addition, local feature representation learning is applied to stabilize the decision boundaries and improve the local feature extraction capabilities for clients. While the existing methods require the collection of auxiliary data or model weights to generate a counterpart, FedClassAvg only requires clients to communicate with a couple of fully connected layers, which is highly communication-efficient. Moreover, FedClassAvg does not require extra optimization problems such as knowledge transfer, which requires intensive computation overhead. We evaluated FedClassAvg through extensive experiments and demonstrated it outperforms the current state-of-the-art algorithms on heterogeneous personalized federated learning tasks.


Personalized Federated Learning through Local Memorization

Federated learning allows clients to collaboratively learn statistical m...

Personalized Federated Learning with Feature Alignment and Classifier Collaboration

Data heterogeneity is one of the most challenging issues in federated le...

Layer-wised Model Aggregation for Personalized Federated Learning

Personalized Federated Learning (pFL) not only can capture the common pr...

Compare Where It Matters: Using Layer-Wise Regularization To Improve Federated Learning on Heterogeneous Data

Federated Learning is a widely adopted method to train neural networks o...

DRIVE: One-bit Distributed Mean Estimation

We consider the problem where n clients transmit d-dimensional real-valu...

Federated Learning on Non-IID Graphs via Structural Knowledge Sharing

Graph neural networks (GNNs) have shown their superiority in modeling gr...

Contrast with Major Classifier Vectors for Federated Medical Relation Extraction with Heterogeneous Label Distribution

Federated medical relation extraction enables multiple clients to train ...

Please sign up or login with your details

Forgot password? Click here to reset