A bootstrap procedure for unsupervised classification based on variable and observation resampling

Abstract

In this thesis, we first investigate the performance of bagged sparse 2-means clustering using a conventional bagging procedure. We next propose a cross bagging procedure, which bootstraps the variables and observations simultaneously, for sparse unsupervised classification problems. Formulated under a general framework, the procedure admits a variety of plugged-in classifiers, which can be determined case by case. Our procedure comprises two main stages. The main goal in the first stage is to collect class probabilities and variable importance. Another round of modified bagging is implemented in the follow-up stage, aiming to re-classify the original learning set so that the final partition is more accurate than that obtained by a single-round bagging. We illustrate the potential applicability of cross bagging by studying its behavior when sparse 2-means clustering is taken to be the sparse unsupervised classifier and k-nearest-neighbor classifier is taken as the supervised classifier. Several simulations are reported to demonstrate the performances of cross bagged sparse 2-means clustering, which are compared with those of sparse 2-means clustering and its bagged version. The results show that, under the aforementioned setting, cross bagged sparse 2-means clustering approximates Bayes rule the best when n is greater or close to p. However, this conclusion may not be true if n is much smaller than p.