Dimension reduction via projection and its applications

Abstract

In the era of big data, high-dimensional data sets are widely encountered in the fields of genomics, time series and machine learning. Projection technique from dimension reduction is efficient to compress number of features and conduct statistical interpretation. With the advancement of modern technology, parametric tensor decomposition and nonparametric angle-based distance are popular projection tools.

In the first part of the thesis, a newly proposed tensor train (TT) decomposition is used to compress parametric subspace of tensor regression. Many existing models for high-dimensional data are based on Tucker decomposition, which has good properties but loses its efficiency in compressing tensors very quickly as the order of tensors increases, say greater than four or five. We propose a modified TT decomposition and then applies it to tensor regression such that a nice statistical interpretation can be obtained. The new tensor regression can well match the data with hierarchical structures. More importantly, the new tensor regression can be easily applied to the case with higher order tensors since TT decomposition can compress the coefficient tensors much more efficiently. The methodology is also extended to tensor autoregression for time series data, and nonasymptotic properties are derived for ordinary least squares estimations of both tensor regression and autoregression. A new algorithm is introduced to search for estimators, and its theoretical justification is also discussed. Theoretical and computational properties of the proposed methodology are verified by simulation studies, and the advantages over existing methods are illustrated by two real examples.

Secondly, a novel projection mean variance (PMV) measure from nonparametric model is used to test the multi-sample hypothesis of equal distributions for univariate or multivariate responses. The proposed PMV measure generalizes the mean variance index using projection technique. The PMV measure yields an analogous variance component decomposition. Using this decomposition, an ANOVA F statistic is derived to test the multi-sample problem. The proposed test is statistically consistent against general alternatives and robust to heavy-tailed data. The test is free of tuning parameters and does not require moment conditions on the response. Our simulation results demonstrate that the PMV test has higher power than classical Wilks-type methods and DISCO test, especially when the dimension of the response is relatively large or the moment conditions required by the DISCO test are violated. We further illustrate our method using empirical analyses of two real data sets.

Lastly, projection quantile correlation (PQC) is proposed to detect quantile dependence between a response and multivariate predictors at a given quantile level. We then use the measure to select grouped predictors that contribute to conditional quantile of the response for high-dimensional data with group structures. Sure independent screening property is established for the group screening method. We illustrate the finite-sample performance of the proposed method through simulations and an application to a data set.