Bootstrap methods for Lasso-type regression estimators

Abstract

We study the distributions of Lasso-type regression estimators in a moving-parameter asymptotic framework, and consider various bootstrap methods for estimating them accordingly. We show, in particular, that the distribution functions of Lasso-type estimators, including even those possessing the oracle properties such as the adaptive Lasso and the SCAD, cannot be consistently estimated by the bootstraps uniformly over the space of the regression parameters, especially when some of the regression coefficients lie close to the origin. Such lack of uniform consistency poses difficulties in practical applications of the bootstraps for making Lasso-based inferences. In the light of this seemingly negative result, we seek, however, to develop criteria for assessing the relative risks, phrased in terms of their uniform consistency properties, of the various bootstrap methods, based on which an optimal bootstrap strategy may be formulated in an adaptive manner. A simulation study is provided to demonstrate the non-normal nature of the distributions of Lasso-type estimators, and to assess the performances of various bootstrap estimates across different values of regression parameters.