Bias-aware model selection for machine learning of doubly robust functionals

by   Yifan Cui, et al.

While model selection is a well-studied topic in parametric and nonparametric regression or density estimation, model selection of possibly high dimensional nuisance parameters in semiparametric problems is far less developed. In this paper, we propose a new model selection framework for making inferences about a finite dimensional functional defined on a semiparametric model, when the latter admits a doubly robust estimating function. The class of such doubly robust functionals is quite large, including many missing data and causal inference problems. Under double robustness, the estimated functional should incur no bias if either of two nuisance parameters is evaluated at the truth while the other spans a large collection of candidate models. We introduce two model selection criteria for bias reduction of functional of interest, each based on a novel definition of pseudo-risk for the functional that embodies this double robustness property and thus may be used to select the candidate model that is nearest to fulfilling this property even when all models are wrong. Both selection criteria have a bias awareness property that selection of one nuisance parameter can be made to compensate for excessive bias due to poor learning of the other nuisance parameter. We establish an oracle property for a multi-fold cross-validation version of the new model selection criteria which states that our empirical criteria perform nearly as well as an oracle with a priori knowledge of the pseudo-risk for each candidate model. We also describe a smooth approximation to the selection criteria which allows for valid post-selection inference. Finally, we perform model selection of a semiparametric estimator of average treatment effect given an ensemble of candidate machine learning methods to account for confounding in a study of right heart catheterization in the ICU of critically ill patients.


page 1

page 2

page 3

page 4


Local asymptotics of cross-validation in least-squares density estimation

In model selection, several types of cross-validation are commonly used ...

Adaptive debiased machine learning using data-driven model selection techniques

Debiased machine learning estimators for nonparametric inference of smoo...

Have we been Naive to Select Machine Learning Models? Noisy Data are here to Stay!

The model selection procedure is usually a single-criterion decision mak...

Semiparametric counterfactual density estimation

Causal effects are often characterized with averages, which can give an ...

Non-Bayesian Post-Model-Selection Estimation as Estimation Under Model Misspecification

In many parameter estimation problems, the exact model is unknown and is...

V-fold cross-validation improved: V-fold penalization

We study the efficiency of V-fold cross-validation (VFCV) for model sele...

Please sign up or login with your details

Forgot password? Click here to reset