Towards Communication-efficient Vertical Federated Learning Training via Cache-enabled Local Updates

by   Fangcheng Fu, et al.
Peking University
Wuhan University

Vertical federated learning (VFL) is an emerging paradigm that allows different parties (e.g., organizations or enterprises) to collaboratively build machine learning models with privacy protection. In the training phase, VFL only exchanges the intermediate statistics, i.e., forward activations and backward derivatives, across parties to compute model gradients. Nevertheless, due to its geo-distributed nature, VFL training usually suffers from the low WAN bandwidth. In this paper, we introduce CELU-VFL, a novel and efficient VFL training framework that exploits the local update technique to reduce the cross-party communication rounds. CELU-VFL caches the stale statistics and reuses them to estimate model gradients without exchanging the ad hoc statistics. Significant techniques are proposed to improve the convergence performance. First, to handle the stochastic variance problem, we propose a uniform sampling strategy to fairly choose the stale statistics for local updates. Second, to harness the errors brought by the staleness, we devise an instance weighting mechanism that measures the reliability of the estimated gradients. Theoretical analysis proves that CELU-VFL achieves a similar sub-linear convergence rate as vanilla VFL training but requires much fewer communication rounds. Empirical results on both public and real-world workloads validate that CELU-VFL can be up to six times faster than the existing works.


Hijack Vertical Federated Learning Models with Adversarial Embedding

Vertical federated learning (VFL) is an emerging paradigm that enables c...

A Communication Efficient Vertical Federated Learning Framework

One critical challenge for applying today's Artificial Intelligence (AI)...

Cross-Silo Federated Learning for Multi-Tier Networks with Vertical and Horizontal Data Partitioning

We consider federated learning in tiered communication networks. Our net...

Achieving Model Fairness in Vertical Federated Learning

Vertical federated learning (VFL), which enables multiple enterprises po...

Communication-Efficient Federated Learning with Compensated Overlap-FedAvg

Petabytes of data are generated each day by emerging Internet of Things ...

Flexible Vertical Federated Learning with Heterogeneous Parties

We propose Flexible Vertical Federated Learning (Flex-VFL), a distribute...

AsySQN: Faster Vertical Federated Learning Algorithms with Better Computation Resource Utilization

Vertical federated learning (VFL) is an effective paradigm of training t...

Please sign up or login with your details

Forgot password? Click here to reset