Towards Instance-adaptive Inference for Federated Learning

by   Chun-Mei Feng, et al.

Federated learning (FL) is a distributed learning paradigm that enables multiple clients to learn a powerful global model by aggregating local training. However, the performance of the global model is often hampered by non-i.i.d. distribution among the clients, requiring extensive efforts to mitigate inter-client data heterogeneity. Going beyond inter-client data heterogeneity, we note that intra-client heterogeneity can also be observed on complex real-world data and seriously deteriorate FL performance. In this paper, we present a novel FL algorithm, i.e., FedIns, to handle intra-client data heterogeneity by enabling instance-adaptive inference in the FL framework. Instead of huge instance-adaptive models, we resort to a parameter-efficient fine-tuning method, i.e., scale and shift deep features (SSF), upon a pre-trained model. Specifically, we first train an SSF pool for each client, and aggregate these SSF pools on the server side, thus still maintaining a low communication cost. To enable instance-adaptive inference, for a given instance, we dynamically find the best-matched SSF subsets from the pool and aggregate them to generate an adaptive SSF specified for the instance, thereby reducing the intra-client as well as the inter-client heterogeneity. Extensive experiments show that our FedIns outperforms state-of-the-art FL algorithms, e.g., a 6.64% improvement against the top-performing method with less than 15% communication cost on Tiny-ImageNet. Our code and models will be publicly released.


Federated Learning on Heterogeneous Data via Adaptive Self-Distillation

Federated Learning (FL) is a machine learning paradigm that enables clie...

Learning Federated Visual Prompt in Null Space for MRI Reconstruction

Federated Magnetic Resonance Imaging (MRI) reconstruction enables multip...

Cooperative Hardware-Prompt Learning for Snapshot Compressive Imaging

Snapshot compressive imaging emerges as a promising technology for acqui...

Federated Learning with Neural Graphical Models

Federated Learning (FL) addresses the need to create models based on pro...

Cross-domain Federated Adaptive Prompt Tuning for CLIP

Federated learning (FL) allows multiple parties to collaboratively train...

One-Time Model Adaptation to Heterogeneous Clients: An Intra-Client and Inter-Image Attention Design

The mainstream workflow of image recognition applications is first train...

Joint Local Relational Augmentation and Global Nash Equilibrium for Federated Learning with Non-IID Data

Federated learning (FL) is a distributed machine learning paradigm that ...

Please sign up or login with your details

Forgot password? Click here to reset