A Unifying Perspective on Multi-Calibration: Unleashing Game Dynamics for Multi-Objective Learning

by   Nika Haghtalab, et al.
berkeley college

We provide a unifying framework for the design and analysis of multi-calibrated and moment-multi-calibrated predictors. Placing the multi-calibration problem in the general setting of multi-objective learning – where learning guarantees must hold simultaneously over a set of distributions and loss functions – we exploit connections to game dynamics to obtain state-of-the-art guarantees for a diverse set of multi-calibration learning problems. In addition to shedding light on existing multi-calibration guarantees, and greatly simplifying their analysis, our approach yields a 1/ϵ^2 improvement in the number of oracle calls compared to the state-of-the-art algorithm of Jung et al. 2021 for learning deterministic moment-calibrated predictors and an exponential improvement in k compared to the state-of-the-art algorithm of Gopalan et al. 2022 for learning a k-class multi-calibrated predictor. Beyond multi-calibration, we use these game dynamics to address existing and emerging considerations in the study of group fairness and multi-distribution learning.


page 1

page 2

page 3

page 4


Scaffolding Sets

Predictors map individual instances in a population to the interval [0,1...

When Does Optimizing a Proper Loss Yield Calibration?

Optimizing proper loss functions is popularly believed to yield predicto...

Causal isotonic calibration for heterogeneous treatment effects

We propose causal isotonic calibration, a novel nonparametric method for...

A Finer Calibration Analysis for Adversarial Robustness

We present a more general analysis of H-calibration for adversarially ro...

"Calibeating": Beating Forecasters at Their Own Game

In order to identify expertise, forecasters should not be tested by thei...

Game-Theoretic Algorithms for Conditional Moment Matching

A variety of problems in econometrics and machine learning, including in...

Please sign up or login with your details

Forgot password? Click here to reset