Combating Data Imbalances in Federated Semi-supervised Learning with Dual Regulators

by   Sikai Bai, et al.

Federated learning has become a popular method to learn from decentralized heterogeneous data. Federated semi-supervised learning (FSSL) emerges to train models from a small fraction of labeled data due to label scarcity on decentralized clients. Existing FSSL methods assume independent and identically distributed (IID) labeled data across clients and consistent class distribution between labeled and unlabeled data within a client. This work studies a more practical and challenging scenario of FSSL, where data distribution is different not only across clients but also within a client between labeled and unlabeled data. To address this challenge, we propose a novel FSSL framework with dual regulators, FedDure. FedDure lifts the previous assumption with a coarse-grained regulator (C-reg) and a fine-grained regulator (F-reg): C-reg regularizes the updating of the local model by tracking the learning effect on labeled data distribution; F-reg learns an adaptive weighting scheme tailored for unlabeled instances in each client. We further formulate the client model training as bi-level optimization that adaptively optimizes the model in the client with two regulators. Theoretically, we show the convergence guarantee of the dual regulators. Empirically, we demonstrate that FedDure is superior to the existing methods across a wide range of settings, notably by more than 11 on CIFAR-10 and CINIC-10 datasets.


page 1

page 2

page 3

page 4


Federated Semi-Supervised Learning with Inter-Client Consistency

While existing federated learning approaches mostly require that clients...

Federated Semi-Supervised Learning with Class Distribution Mismatch

Many existing federated learning (FL) algorithms are designed for superv...

Dual Class-Aware Contrastive Federated Semi-Supervised Learning

Federated semi-supervised learning (FSSL), facilitates labeled clients a...

Federated Semi-Supervised Learning with Annotation Heterogeneity

Federated Semi-Supervised Learning (FSSL) aims to learn a global model f...

Towards Unbiased Training in Federated Open-world Semi-supervised Learning

Federated Semi-supervised Learning (FedSSL) has emerged as a new paradig...

Addressing Class Variable Imbalance in Federated Semi-supervised Learning

Federated Semi-supervised Learning (FSSL) combines techniques from both ...

FedCon: A Contrastive Framework for Federated Semi-Supervised Learning

Federated Semi-Supervised Learning (FedSSL) has gained rising attention ...

Please sign up or login with your details

Forgot password? Click here to reset