Joint Local Relational Augmentation and Global Nash Equilibrium for Federated Learning with Non-IID Data

by   Xinting Liao, et al.

Federated learning (FL) is a distributed machine learning paradigm that needs collaboration between a server and a series of clients with decentralized data. To make FL effective in real-world applications, existing work devotes to improving the modeling of decentralized data with non-independent and identical distributions (non-IID). In non-IID settings, there are intra-client inconsistency that comes from the imbalanced data modeling, and inter-client inconsistency among heterogeneous client distributions, which not only hinders sufficient representation of the minority data, but also brings discrepant model deviations. However, previous work overlooks to tackle the above two coupling inconsistencies together. In this work, we propose FedRANE, which consists of two main modules, i.e., local relational augmentation (LRA) and global Nash equilibrium (GNE), to resolve intra- and inter-client inconsistency simultaneously. Specifically, in each client, LRA mines the similarity relations among different data samples and enhances the minority sample representations with their neighbors using attentive message passing. In server, GNE reaches an agreement among inconsistent and discrepant model deviations from clients to server, which encourages the global model to update in the direction of global optimum without breaking down the clients optimization toward their local optimums. We conduct extensive experiments on four benchmark datasets to show the superiority of FedRANE in enhancing the performance of FL with non-IID data.


page 4

page 7


HyperFed: Hyperbolic Prototypes Exploration with Consistent Aggregation for Non-IID Data in Federated Learning

Federated learning (FL) collaboratively models user data in a decentrali...

FRAug: Tackling Federated Learning with Non-IID Features via Representation Augmentation

Federated Learning (FL) is a decentralized learning paradigm in which mu...

M3FGM:a node masking and multi-granularity message passing-based federated graph model for spatial-temporal data prediction

Researchers are solving the challenges of spatial-temporal prediction by...

Towards Instance-adaptive Inference for Federated Learning

Federated learning (FL) is a distributed learning paradigm that enables ...

Federated Learning for Non-IID Data via Client Variance Reduction and Adaptive Server Update

Federated learning (FL) is an emerging technique used to collaboratively...

Learning Private Neural Language Modeling with Attentive Aggregation

Mobile keyboard suggestion is typically regarded as a word-level languag...

One-Time Model Adaptation to Heterogeneous Clients: An Intra-Client and Inter-Image Attention Design

The mainstream workflow of image recognition applications is first train...

Please sign up or login with your details

Forgot password? Click here to reset