Regression penalizing deviations from additivity
Reference
Studer M., Seifert B., Gasser T. Nonparametric regression penalizing deviations from additivity. Ann. Stat. 33 (3): 1295-1329, 2005.
Abstract
Due to the curse of dimensionality, estimation in a multidimensional nonparametric regression model is in general not feasible. Hence, additional restrictions are introduced, and the additive model takes a prominent place. The restrictions imposed can lead to serious bias. Here, a new estimator is proposed which allows penalizing the nonadditive part of a regression function. This offers a smooth choice between the full and the additive model. As a byproduct, this penalty leads to a regularization in sparse regions. If the additive model does not hold, a small penalty introduces an additional bias compared to the full model which is compensated by the reduced bias due to using smaller bandwidths. For increasing penalties, this estimator converges to the additive smooth backfitting estimator of Mammen, Linton, and Nielsen (1999). The structure of the estimator is investigated and two algorithms are provided. A proposal for selection of tuning parameters is made and the respective properties are studied. Finally, a finite sample evaluation is performed for simulated and the ozone data.
R packages
In the software, the term `additive' denotes a multivariate function r(x1,...,xd) which may be written as sum of all univariate terms r1(x1)+...+ rd(xd) plus optionally some bivariate interaction terms rk,l(xk,xl). The term `univariate additive' denotes an additive function without bivariate terms. Keep in mind that including bivariate terms and using a fine output grid may be computationally infeasible.
- Here is the R code for reproducing tables and figures in the paper. It depends on the R package NonAddPenalty below (computation may take a long time).
- NonAddPenalty calculates a multivariate local linear estimator with penalty on the nonadditive part.
- SBF2 estimates an additive regression function with penalty on the bivariate interaction terms. (Univariate additive components are not shrinked.)
The above Software is written in C and the interfaces work for Splus and R. The license is BSD.