Federated Conformal Predictors for Distributed Uncertainty Quantification

by   Charles Lu, et al.

Conformal prediction is emerging as a popular paradigm for providing rigorous uncertainty quantification in machine learning since it can be easily applied as a post-processing step to already trained models. In this paper, we extend conformal prediction to the federated learning setting. The main challenge we face is data heterogeneity across the clients – this violates the fundamental tenet of exchangeability required for conformal prediction. We propose a weaker notion of partial exchangeability, better suited to the FL setting, and use it to develop the Federated Conformal Prediction (FCP) framework. We show FCP enjoys rigorous theoretical guarantees and excellent empirical performance on several computer vision and medical imaging datasets. Our results demonstrate a practical approach to incorporating meaningful uncertainty quantification in distributed and heterogeneous environments. We provide code used in our experiments <https://github.com/clu5/federated-conformal>.


page 8

page 19

page 20

page 23


Conformal Prediction for Federated Uncertainty Quantification Under Label Shift

Federated Learning (FL) is a machine learning framework where many clien...

FedPop: A Bayesian Approach for Personalised Federated Learning

Personalised federated learning (FL) aims at collaboratively learning a ...

Privacy Preserving Bayesian Federated Learning in Heterogeneous Settings

In several practical applications of federated learning (FL), the client...

URL: A Representation Learning Benchmark for Transferable Uncertainty Estimates

Representation learning has significantly driven the field to develop pr...

On Convergence of Federated Averaging Langevin Dynamics

We propose a federated averaging Langevin algorithm (FA-LD) for uncertai...

Distribution-Free Federated Learning with Conformal Predictions

Federated learning has attracted considerable interest for collaborative...

Conformal Prediction for Trustworthy Detection of Railway Signals

We present an application of conformal prediction, a form of uncertainty...

Please sign up or login with your details

Forgot password? Click here to reset