Splitfed learning without client-side synchronization: Analyzing client-side split network portion size to overall performance

by   Praveen Joshi, et al.

Federated Learning (FL), Split Learning (SL), and SplitFed Learning (SFL) are three recent developments in distributed machine learning that are gaining attention due to their ability to preserve the privacy of raw data. Thus, they are widely applicable in various domains where data is sensitive, such as large-scale medical image classification, internet-of-medical-things, and cross-organization phishing email detection. SFL is developed on the confluence point of FL and SL. It brings the best of FL and SL by providing parallel client-side machine learning model updates from the FL paradigm and a higher level of model privacy (while training) by splitting the model between the clients and server coming from SL. However, SFL has communication and computation overhead at the client-side due to the requirement of client-side model synchronization. For the resource-constrained client-side, removal of such requirements is required to gain efficiency in the learning. In this regard, this paper studies SFL without client-side model synchronization. The resulting architecture is known as Multi-head Split Learning. Our empirical studies considering the ResNet18 model on MNIST data under IID data distribution among distributed clients find that Multi-head Split Learning is feasible. Its performance is comparable to the SFL. Moreover, SFL provides only 1 further strengthen our results, we study the Multi-head Split Learning with various client-side model portions and its impact on the overall performance. To this end, our results find a minimal impact on the overall performance of the model.


page 1

page 2

page 3

page 4


SplitFed: When Federated Learning Meets Split Learning

Federated learning (FL) and split learning (SL) are two recent distribut...

SplitFed resilience to packet loss: Where to split, that is the question

Decentralized machine learning has broadened its scope recently with the...

Communication and Storage Efficient Federated Split Learning

Federated learning (FL) is a popular distributed machine learning (ML) p...

Advancements of federated learning towards privacy preservation: from federated learning to split learning

In the distributed collaborative machine learning (DCML) paradigm, feder...

Federated Split Vision Transformer for COVID-19 CXR Diagnosis using Task-Agnostic Training

Federated learning, which shares the weights of the neural network acros...

Quality-Adaptive Split-Federated Learning for Segmenting Medical Images with Inaccurate Annotations

SplitFed Learning, a combination of Federated and Split Learning (FL and...

Communication and Computation Reduction for Split Learning using Asynchronous Training

Split learning is a promising privacy-preserving distributed learning sc...

Please sign up or login with your details

Forgot password? Click here to reset