Characterization of all solutions for undersampled uncorrelated linear discriminant analysis problems

Abstract

In this paper the uncorrelated linear discriminant analysis (ULDA) for undersampled problems is studied. The main contributions of the present work include the following: (i) all solutions of the optimization problem used for establishing the ULDA are parameterized explicitly; (ii) the optimal solutions among all solutions of the corresponding optimization problem are characterized in terms of both the ratio of between-class distance to within-class distance and the maximum likelihood classification, and it is proved that these optimal solutions are exactly the solutions of the corresponding optimization problem with minimum Frobenius norm, also minimum nuclear norm; these properties provide a good mathematical justification for preferring the minimum-norm transformation over other possible solutions as the optimal transformation in ULDA; (iii) explicit necessary and sufficient conditions are provided to ensure that these minimal solutions lead to a larger ratio of between-class distance to within-class distance, thereby achieving larger discrimination in the reduced subspace than that in the original data space, and our numerical experiments show that these necessary and sufficient conditions hold true generally. Furthermore, a new and fast ULDA algorithm is developed, which is eigendecomposition-free and SVD-free, and its effectiveness is demonstrated by some real-world data sets. © 2011 Society for Industrial and Applied Mathematics.