Statistical Learnability of Generalized Additive Models based on Total Variation Regularization
share
A generalized additive model (GAM, Hastie and Tibshirani (1987)) is a nonparametric model by the sum of univariate functions with respect to each explanatory variable, i.e., f( x) = ∑ f_j(x_j), where x_j∈R is j-th component of a sample x∈R^p. In this paper, we introduce the total variation (TV) of a function as a measure of the complexity of functions in L^1_ c(R)-space. Our analysis shows that a GAM based on TV-regularization exhibits a Rademacher complexity of O(√( p/m)), which is tight in terms of both m and p in the agnostic case of the classification problem. In result, we obtain generalization error bounds for finite samples according to work by Bartlett and Mandelson (2002).
READ FULL TEXT