Distributional Robustness of K-class Estimators and the PULSE

by   Martin Emil Jakobsen, et al.

In causal settings, such as instrumental variable settings, it is well known that estimators based on ordinary least squares (OLS) can yield biased and non-consistent estimates of the causal parameters. This is partially overcome by two-stage least squares (TSLS) estimators. These are, under weak assumptions, consistent but do not have desirable finite sample properties: in many models, for example, they do not have finite moments. The set of K-class estimators can be seen as a non-linear interpolation between OLS and TSLS and are known to have improved finite sample properties. Recently, in causal discovery, invariance properties such as the moment criterion which TSLS estimators leverage have been exploited for causal structure learning: e.g., in cases, where the causal parameter is not identifiable, some structure of the non-zero components may be identified, and coverage guarantees are available. Subsequently, anchor regression has been proposed to trade-off invariance and predictability. The resulting estimator is shown to have optimal predictive performance under bounded shift interventions. In this paper, we show that the concepts of anchor regression and K-class estimators are closely related. Establishing this connection comes with two benefits: (1) It enables us to prove robustness properties for existing K-class estimators when considering distributional shifts. And, (2), we propose a novel estimator in instrumental variable settings by minimizing the mean squared prediction error subject to the constraint that the estimator lies in an asymptotically valid confidence region of the causal parameter. We call this estimator PULSE (p-uncorrelated least squares estimator) and show that it can be computed efficiently, even though the underlying optimization problem is non-convex. We further prove that it is consistent.


page 1

page 2

page 3

page 4


Causality and Generalizability: Identifiability and Learning Methods

This PhD thesis contains several contributions to the field of statistic...

On the implied weights of linear regression for causal inference

In this paper, we derive and analyze the implied weights of linear regre...

Causal Regularization: On the trade-off between in-sample risk and out-of-sample risk guarantees

In recent decades, a number of ways of dealing with causality in practic...

Nonlinear Causal Discovery via Kernel Anchor Regression

Learning causal relationships is a fundamental problem in science. Ancho...

Distributional Anchor Regression

Prediction models often fail if train and test data do not stem from the...

Exploiting Independent Instruments: Identification and Distribution Generalization

Instrumental variable models allow us to identify a causal function betw...

Optimal multi-environment causal regularization

In this manuscript we derive the optimal out-of-sample causal predictor ...

Please sign up or login with your details

Forgot password? Click here to reset