Feature Extraction by Non-Parametric Mutual Information Maximization

Kari Torkkola; 3(Mar):1415-1438, 2003.

Abstract

We present a method for learning discriminative feature transforms using as criterion the mutual information between class labels and transformed features. Instead of a commonly used mutual information measure based on Kullback-Leibler divergence, we use a quadratic divergence measure, which allows us to make an efficient non-parametric implementation and requires no prior assumptions about class densities. In addition to linear transforms, we also discuss nonlinear transforms that are implemented as radial basis function networks. Extensions to reduce the computational complexity are also presented, and a comparison to greedy feature selection is made.

[abs] [pdf] [ps.gz] [ps] [demos]