线性回归模型中的变量选择:选择最好的子集并不总是最好的选择
Moritz Hanke1, Louis Dijkstra1, Ronja Foraita1
1Department of Biometry and Data Management, Leibniz Institute for Prevention Research and Epidemiology - BIPS, Bremen, Germany.
Biometrical journal. Biometrische Zeitschrift
|August 29, 2023
概括
最好的子集选择 (BSS) 不总是优于在线性回归中识别预测器. 像Lasso和Elastic net这样的替代品通常表现更好,特别是在相关变量方面.
科学领域:
- 统计 统计 统计 统计
- 机器学习 机器学习
- 计算统计学 计算统计学
背景情况:
- 变量选择对于可解释的线性回归模型至关重要.
- 最好的子集选择 (BSS) 在理论上是最佳的,但在计算上是昂贵的.
- 拉索和弹性网是受欢迎的替代品,特别是在高维数据中.
研究的目的:
- 进行变量选择方法的中立比较.
- 评估BSS与前进阶段选择 (FSS),拉索和弹性网 (Enet) 的性能.
- 在具有挑战性的条件下评估方法,包括高维度,不同的信号噪声比率和预测器相关性.
主要方法:
- 我们使用模拟来比较BSS,FSS,Lasso和Enet.
- 性能主要以最好的F1得分来衡量.
- 还使用了替代性能测量和参数调整的实际标准.
主要成果:
- 只有在高信号噪声比设置中,BSS在无关变量设置中可靠地超过了其他方法.
- 前进阶段选择 (FSS) 几乎与BSS.几乎完全相同.
- 弹性网 (Enet) 显示出与相关变量相对应的速度和性能优势.
结论:
- 质疑BSS总是变量选择的最佳选择的假设.
- 对于相关预测器,Enet是一个更快,通常更好的实用替代方案.
- 该研究为常见变量选择技术的比较性能提供了新的见解.
相关概念视频
Multiple Regression
3.0K
Multiple regression assesses a linear relationship between one response or dependent variable and two or more independent variables. It has many practical applications.
Farmers can use multiple regression to determine the crop yield based on more than one factor, such as water availability, fertilizer, soil properties, etc. Here, the crop yield is the response or dependent variable as it depends on the other independent variables. The analysis requires the construction of a scatter plot...
Farmers can use multiple regression to determine the crop yield based on more than one factor, such as water availability, fertilizer, soil properties, etc. Here, the crop yield is the response or dependent variable as it depends on the other independent variables. The analysis requires the construction of a scatter plot...
3.0K
Mechanistic Models: Compartment Models in Algorithms for Numerical Problem Solving
79
Mechanistic models play a crucial role in algorithms for numerical problem-solving, particularly in nonlinear mixed effects modeling (NMEM). These models aim to minimize specific objective functions by evaluating various parameter estimates, leading to the development of systematic algorithms. In some cases, linearization techniques approximate the model using linear equations.
In individual population analyses, different algorithms are employed, such as Cauchy's method, which uses a...
In individual population analyses, different algorithms are employed, such as Cauchy's method, which uses a...
79
Choosing Between z and t Distribution
2.8K
The z and the Student t distribution estimate the population mean using the sample mean and standard deviation. However, to decide which distribution to use for a calculation, one needs to determine the sample size, the nature of the distribution, and whether the population standard deviation is known. If the population standard deviation is known and the population is normally distributed, or if the sample size is greater than 30, the z distribution is preferred. The Student t distribution is...
2.8K
Regression Analysis
5.8K
Regression analysis is a statistical tool that describes a mathematical relationship between a dependent variable and one or more independent variables.
In regression analysis, a regression equation is determined based on the line of best fit– a line that best fits the data points plotted in a graph. This line is also called the regression line. The algebraic equation for the regression line is called the regression equation. It is represented as:
In regression analysis, a regression equation is determined based on the line of best fit– a line that best fits the data points plotted in a graph. This line is also called the regression line. The algebraic equation for the regression line is called the regression equation. It is represented as:
5.8K
Types of Selection
40.6K
Natural selection influences the frequencies of particular alleles and phenotypes within populations in several different ways. Primarily, natural selection can be directional, stabilizing, or disruptive. Directional selection favors one extreme trait and shifts the population towards that phenotype while selecting against individuals displaying alternate traits. Stabilizing selection favors an intermediate trait with a narrow range of variation. Deviation from the optimal phenotype towards an...
40.6K
Pharmacokinetic Models: Comparison and Selection Criterion
104
Physiological and compartmental models are valuable tools used in studying biological systems. These models rely on differential equations to maintain mass balance within the system, ensuring an accurate representation of the dynamic processes at play.
Physiological models take a detailed approach by considering specific molecular processes. They can predict drug distribution, metabolism, and elimination changes, providing a comprehensive understanding of how drugs interact with the body.
Physiological models take a detailed approach by considering specific molecular processes. They can predict drug distribution, metabolism, and elimination changes, providing a comprehensive understanding of how drugs interact with the body.
104


