One-Time Model Adaptation to Heterogeneous Clients: An Intra-Client and Inter-Image Attention Design

by   Yikai Yan, et al.

The mainstream workflow of image recognition applications is first training one global model on the cloud for a wide range of classes and then serving numerous clients, each with heterogeneous images from a small subset of classes to be recognized. From the cloud-client discrepancies on the range of image classes, the recognition model is desired to have strong adaptiveness, intuitively by concentrating the focus on each individual client's local dynamic class subset, while incurring negligible overhead. In this work, we propose to plug a new intra-client and inter-image attention (ICIIA) module into existing backbone recognition models, requiring only one-time cloud-based training to be client-adaptive. In particular, given a target image from a certain client, ICIIA introduces multi-head self-attention to retrieve relevant images from the client's historical unlabeled images, thereby calibrating the focus and the recognition result. Further considering that ICIIA's overhead is dominated by linear projection, we propose partitioned linear projection with feature shuffling for replacement and allow increasing the number of partitions to dramatically improve efficiency without scarifying too much accuracy. We finally evaluate ICIIA using 3 different recognition tasks with 9 backbone models over 5 representative datasets. Extensive evaluation results demonstrate the effectiveness and efficiency of ICIIA. Specifically, for ImageNet-1K with the backbone models of MobileNetV3-L and Swin-B, ICIIA can improve the testing accuracy to 83.37 0.02


page 1

page 3

page 4


Federated Minimax Optimization with Client Heterogeneity

Minimax optimization has seen a surge in interest with the advent of mod...

Federated Incremental Semantic Segmentation

Federated learning-based semantic segmentation (FSS) has drawn widesprea...

Towards Instance-adaptive Inference for Federated Learning

Federated learning (FL) is a distributed learning paradigm that enables ...

ConcurORAM: High-Throughput Stateless Parallel Multi-Client ORAM

ConcurORAM is a parallel, multi-client ORAM that eliminates waiting for ...

Learning to Collaborate

In this paper, we focus on effective learning over a collaborative resea...

Joint Local Relational Augmentation and Global Nash Equilibrium for Federated Learning with Non-IID Data

Federated learning (FL) is a distributed machine learning paradigm that ...

Please sign up or login with your details

Forgot password? Click here to reset