A few theoretical results for Laplace and arctan penalized ordinary least squares linear regression estimators

Two new nonconvex penalty functions - Laplace and arctan - were recently introduced in the literature to obtain sparse models for high-dimensional statistical problems. In this paper, we study the theoretical properties of Laplace and arctan penalized ordinary least squares linear regression models. We first illustrate the near-unbiasedness of the nonzero regression weights obtained by the new penalty functions, in the orthonormal design case. In the general design case, we present theoretical results in two asymptotic settings: (a) the number of features, $$ fixed, but the sample size, $$ , and (b) both $$ and $$ tend to infinity. The theoretical results shed light onto the differences between the solutions based on the new penalty functions and those based on existing convex and nonconvex Bridge penalty functions. Our theory also shows that both Laplace and arctan penalties satisfy the oracle property. Finally, we also present results from a brief simulations study illustrating the performance of Laplace and arctan penalties based on the gradient descent optimization algorithm.