HyperFed: Hyperbolic Prototypes Exploration with Consistent Aggregation for Non-IID Data in Federated Learning

07/26/2023
by   Xinting Liao, et al.
Zhejiang University
Fuzhou University
0

Federated learning (FL) collaboratively models user data in a decentralized way. However, in the real world, non-identical and independent data distributions (non-IID) among clients hinder the performance of FL due to three issues, i.e., (1) the class statistics shifting, (2) the insufficient hierarchical information utilization, and (3) the inconsistency in aggregating clients. To address the above issues, we propose HyperFed which contains three main modules, i.e., hyperbolic prototype Tammes initialization (HPTI), hyperbolic prototype learning (HPL), and consistent aggregation (CA). Firstly, HPTI in the server constructs uniformly distributed and fixed class prototypes, and shares them with clients to match class statistics, further guiding consistent feature representation for local clients. Secondly, HPL in each client captures the hierarchical information in local data with the supervision of shared class prototypes in the hyperbolic model space. Additionally, CA in the server mitigates the impact of the inconsistent deviations from clients to server. Extensive studies of four datasets prove that HyperFed is effective in enhancing the performance of FL under the non-IID set.

READ FULL TEXT

page 10

page 11

08/17/2023

Joint Local Relational Augmentation and Global Nash Equilibrium for Federated Learning with Non-IID Data

Federated learning (FL) is a distributed machine learning paradigm that ...
08/14/2023

Federated Classification in Hyperbolic Spaces via Secure Aggregation of Convex Hulls

Hierarchical and tree-like data sets arise in many applications, includi...
07/20/2023

Boosting Federated Learning Convergence with Prototype Regularization

As a distributed machine learning technique, federated learning (FL) req...
05/12/2022

Secure Aggregation for Federated Learning in Flower

Federated Learning (FL) allows parties to learn a shared prediction mode...
10/17/2020

Secure Weighted Aggregation in Federated Learning

Federated learning (FL) schemes enable multiple clients to jointly solve...
04/01/2023

MP-FedCL: Multi-Prototype Federated Contrastive Learning for Edge Intelligence

Federated learning-assisted edge intelligence enables privacy protection...

Please sign up or login with your details

Forgot password? Click here to reset