Sufficient Dimension Reduction via Squared-loss Mutual Information Estimation

Taiji Suzuki, Masashi Sugiyama ; JMLR W&CP 9:804-811, 2010.

Abstract

The goal of sufficient dimension reduction in supervised learning is to find the lowdimensional subspace of input features that is sufficient for predicting output values. In this paper, we propose a novel sufficient dimension reduction method using a squaredloss variant of mutual information as a dependency measure. We utilize an analytic approximator of squared-loss mutual information based on density ratio estimation, which is shown to possess suitable convergence properties. We then develop a natural gradient algorithm for sufficient subspace search. Numerical experiments show that the proposed method compares favorably with existing dimension reduction approaches.