Canoe : A System for Collaborative Learning for Neural Nets

08/27/2021
by   Harshit Daga, et al.
16

For highly distributed environments such as edge computing, collaborative learning approaches eschew the dependence on a global, shared model, in favor of models tailored for each location. Creating tailored models for individual learning contexts reduces the amount of data transfer, while collaboration among peers provides acceptable model performance. Collaboration assumes, however, the availability of knowledge transfer mechanisms, which are not trivial for deep learning models where knowledge isn't easily attributed to precise model slices. We present Canoe - a framework that facilitates knowledge transfer for neural networks. Canoe provides new system support for dynamically extracting significant parameters from a helper node's neural network and uses this with a multi-model boosting-based approach to improve the predictive performance of the target node. The evaluation of Canoe with different PyTorch and TensorFlow neural network models demonstrates that the knowledge transfer mechanism improves the model's adaptiveness to changes up to 3.5X compared to learning in isolation, while affording several magnitudes reduction in data movement costs compared to federated learning.

READ FULL TEXT

page 1

page 2

page 3

page 4

12/18/2018

Multi-objective Evolutionary Federated Learning

Federated learning is an emerging technique used to prevent the leakage ...
10/27/2018

Fabrik: An Online Collaborative Neural Network Editor

We present Fabrik, an online neural network editor that provides tools t...
06/16/2021

QuantumFed: A Federated Learning Framework for Collaborative Quantum Training

With the fast development of quantum computing and deep learning, quantu...
12/03/2019

ELFISH: Resource-Aware Federated Learning on Heterogeneous Edge Devices

In this work, we propose ELFISH - a resource-aware federated learning fr...
04/03/2021

Knowledge Distillation For Wireless Edge Learning

In this paper, we propose a framework for predicting frame errors in the...
09/10/2019

Knowledge Transfer Graph for Deep Collaborative Learning

We propose Deep Collaborative Learning (DCL), which is a method that inc...
07/14/2021

Federated Mixture of Experts

Federated learning (FL) has emerged as the predominant approach for coll...

Please sign up or login with your details

Forgot password? Click here to reset