A Regularization Approach to Nonlinear Variable Selection

Lorenzo Rosasco, Matteo Santoro, Sofia Mosci, Alessandro Verri, Silvia Villa ; JMLR W&CP 9:653-660, 2010.

Abstract

In this paper we consider a regularization approach to variable selection when the regression function depends nonlinearly on a few input variables. The proposed method is based on a regularized least square estimator penalizing large values of the partial derivatives. An efficient iterative procedure is proposed to solve the underlying variational problem, and its convergence is proved. The empirical properties of the obtained estimator are tested both for prediction and variable selection. The algorithm compares favorably to more standard ridge regression and L1 regularization schemes.