ERM and RERM are optimal estimators for regression problems when malicious outliers corrupt the labels
share
We study Empirical Risk Minimizers (ERM) and Regularized Empirical Risk Minimizers (RERM) for regression problems with convex and L-Lipschitz loss functions. We consider a setting where | O| malicious outliers may contaminate the labels. In that case, we show that the L_2-error rate is bounded by r_N + L | O|/N, where N is the total number of observations and r_N is the L_2-error rate in the non-contaminated setting. When r_N is minimax-rate-optimal in a non-contaminated setting, the rate r_N + L| O|/N is also minimax-rate-optimal when | O| outliers contaminate the label. The main results of the paper can be used for many non-regularized and regularized procedures under weak assumptions on the noise. For instance, we present results for Huber's M-estimators (without penalization or regularized by the ℓ_1-norm) and for general regularized learning problems in reproducible kernel Hilbert spaces.
READ FULL TEXT
