Related Experiment Video
Updated: May 27, 2026
07:35
Selecting Multiple Biomarker Subsets with Similarly Effective Binary Classification Performances
Published on: October 11, 2018
COORDINATE DESCENT ALGORITHMS FOR NONCONVEX PENALIZED REGRESSION, WITH APPLICATIONS TO BIOLOGICAL FEATURE SELECTION
1Department of Biostatistics, Department of Statistics, University of Kentucky, 121 Washington Ave., Room 203C, Lexington, Kentucky 40536-0003, USA. Department of Statistics, and Actuarial Sciences, Department of Biostatistics, University of Iowa, 241 Schaeffer Hall, Iowa City, Iowa 52242, USA.
The Annals of Applied Statistics
|November 15, 2011
Summary
Coordinate descent algorithms offer a faster and more stable approach for variable selection using nonconvex penalties like the minimax concave penalty (MCP) and smoothly clipped absolute deviation (SCAD). These methods are effective alternatives to the lasso, with MCP showing particular promise.
Area of Science:
- Statistics
- Machine Learning
- Computational Statistics
Background:
- Nonconvex penalty functions, such as the smoothly clipped absolute deviation (SCAD) and minimax concave penalty (MCP), offer theoretical advantages for variable selection.
- However, fitting models with these penalties presents challenges, often leading to unstable solutions.
Purpose of the Study:
- To demonstrate the efficacy of coordinate descent algorithms for fitting models with nonconvex penalties.
- To establish the theoretical convergence properties of these algorithms.
- To compare their performance against existing methods.
Main Methods:
- Application of coordinate descent algorithms for model fitting with SCAD and MCP penalties.
- Development and application of convexity diagnostics to identify locally convex regions.
- Comparative analysis with existing variable selection techniques, including the lasso.
Main Results:
- Coordinate descent algorithms provide a stable and significantly faster method for fitting models with nonconvex penalties.
- Convexity diagnostics effectively identify regions of local convexity.
- Simulation studies and data examples show MCP and SCAD as viable alternatives to the lasso, with MCP often being the preferred choice.
Conclusions:
- Coordinate descent algorithms are a powerful tool for variable selection using nonconvex penalties.
- MCP and SCAD, particularly MCP, represent valuable alternatives to the lasso in various applications.
- The integration of convexity diagnostics enhances the reliability of these methods.
Related Concept Videos
01:29Mechanistic Models: Compartment Models in Algorithms for Numerical Problem Solving
Mechanistic models play a crucial role in algorithms for numerical problem-solving, particularly in nonlinear mixed effects modeling (NMEM). These models aim to minimize specific objective functions by evaluating various parameter estimates, leading to the development of systematic algorithms. In some cases, linearization techniques approximate the model using linear equations.
In individual population analyses, different algorithms are employed, such as Cauchy's method, which uses a...
In individual population analyses, different algorithms are employed, such as Cauchy's method, which uses a...
01:52Regression Toward the Mean
Regression toward the mean (“RTM”) is a phenomenon in which extremely high or low values—for example, and individual’s blood pressure at a particular moment—appear closer to a group’s average upon remeasuring. Although this statistical peculiarity is the result of random error and chance, it has been problematic across various medical, scientific, financial and psychological applications. In particular, RTM, if not taken into account, can interfere when researchers try to extrapolate results...
01:21Frequency-dependent Selection
When the fitness of a trait is influenced by how common it is (i.e., its frequency) relative to different traits within a population, this is referred to as frequency-dependent selection. Frequency-dependent selection may occur between species or within a single species. This type of selection can either be positive—with more common phenotypes having higher fitness—or negative, with rarer phenotypes conferring increased fitness.
01:29Application of Nonlinear Inequalities
A nonlinear inequality describes a comparison involving an expression that curves or behaves more complexly than a straight line. These inequalities often appear in forms that include squares, products, or variables in the denominator.To solve such an inequality, one starts by rewriting it so that zero appears on one side. For example, the inequality: can be factored as: This form makes it easier to identify the values that cause the expression to equal zero. In this case, the key values are 3...
01:29Application of Linearization and Approximation
A drone flying through complex terrain often relies on more than one sensing method to estimate small changes in altitude. Along with direct measurements, air pressure provides a useful indirect indicator of vertical movement. Atmospheric pressure decreases as altitude increases, and this relationship is commonly described using an exponential model. Although accurate, converting pressure measurements into altitude values requires calculations that are too complex to perform repeatedly during...
01:20Calibration Curves: Linear Least Squares
A calibration curve is a plot of the instrument's response against a series of known concentrations of a substance. This curve is used to set the instrument response levels, using the substance and its concentrations as standards. Alternatively, or additionally, an equation is fitted to the calibration curve plot and subsequently used to calculate the unknown concentrations of other samples reliably.
For data that follow a straight line, the standard method for fitting is the linear...
For data that follow a straight line, the standard method for fitting is the linear...