Federated Bayesian Neural Regression: A Scalable Global Federated Gaussian Process

by   Haolin Yu, et al.

In typical scenarios where the Federated Learning (FL) framework applies, it is common for clients to have insufficient training data to produce an accurate model. Thus, models that provide not only point estimations, but also some notion of confidence are beneficial. Gaussian Process (GP) is a powerful Bayesian model that comes with naturally well-calibrated variance estimations. However, it is challenging to learn a stand-alone global GP since merging local kernels leads to privacy leakage. To preserve privacy, previous works that consider federated GPs avoid learning a global model by focusing on the personalized setting or learning an ensemble of local models. We present Federated Bayesian Neural Regression (FedBNR), an algorithm that learns a scalable stand-alone global federated GP that respects clients' privacy. We incorporate deep kernel learning and random features for scalability by defining a unifying random kernel. We show this random kernel can recover any stationary kernel and many non-stationary kernels. We then derive a principled approach of learning a global predictive model as if all client data is centralized. We also learn global kernels with knowledge distillation methods for non-identically and independently distributed (non-i.i.d.) clients. Experiments are conducted on real-world regression datasets and show statistically significant improvements compared to other federated GP models.


page 1

page 2

page 3

page 4


Kernel-based Federated Learning with Personalization

We consider federated learning with personalization, where in addition t...

Federated Gaussian Process: Convergence, Automatic Personalization and Multi-fidelity Modeling

In this paper, we propose : a Federated Gaussian process (𝒢𝒫) regression...

Expressive Priors in Bayesian Neural Networks: Kernel Combinations and Periodic Functions

A simple, flexible approach to creating expressive priors in Gaussian pr...

Personalized Federated Learning with Gaussian Processes

Federated learning aims to learn a global model that performs well on cl...

Privacy Amplification via Random Participation in Federated Learning

Running a randomized algorithm on a subsampled dataset instead of the en...

Federated Learning with Neural Graphical Models

Federated Learning (FL) addresses the need to create models based on pro...

Please sign up or login with your details

Forgot password? Click here to reset