Characterization of a Family of Algorithms for Generalized Discriminant Analysis on Undersampled Problems
Jieping Ye; 6(Apr):483--502, 2005.
AbstractA generalized discriminant analysis based on a new optimization criterion is presented. The criterion extends the optimization criteria of the classical Linear Discriminant Analysis (LDA) when the scatter matrices are singular. An efficient algorithm for the new optimization problem is presented.
The solutions to the
proposed criterion form a family of algorithms for generalized LDA,
which can be characterized in a closed form. We study two specific
algorithms, namely Uncorrelated LDA (ULDA) and Orthogonal LDA (OLDA).
ULDA was previously proposed for feature extraction and dimension
reduction, whereas OLDA is a novel algorithm proposed in this paper.
The features in the reduced space of ULDA are uncorrelated, while
the discriminant vectors of OLDA are orthogonal to each other. We
have conducted a comparative study on a variety of real-world data
sets to evaluate ULDA and OLDA in terms of classification accuracy.