通过分析其重量和梯度概况来估计特征的重要性
Ho Tung Jeremy Chan1,2, Eduardo Veas3,4
1Interactive System and Data Science, Graz University of Technology, 8010, Graz, Austria. jchan@know-center.at.
Scientific reports
|October 9, 2024
概括
这项研究通过分析梯度下降,为神经网络引入了一种新的特征选择方法. 它有助于识别重要的特征,减少模型复杂性和培训时间.
科学领域:
- 机器学习 机器学习
- 人工智能的人工智能
- 数据科学数据科学数据科学
背景情况:
- 了解特征的重要性对于高效的神经网络建模至关重要.
- 在神经网络中选择特征的现有方法在解释性上有局限性.
- 识别冗余数据可以提高模型的复杂性,训练时间和内存使用量.
研究的目的:
- 提出一种扩展的方法来估计神经网络中的特征意义.
- 分析双向层的梯度下降,以进行特征重要性分析.
- 通过了解特征重要性层次结构来实现特征排名和选择.
主要方法:
- 在神经网络模型中分析双向层的梯度下降.
- 在训练期间分析重量和梯度的变化,以确定特征的重要性.
- 将方法转移到现有的完全或部分训练模型进行分析.
主要成果:
- 使用基准数据集 (MNIST, scikit-feat),模拟和现实数据的实证演示.
- 根据现有情况对地面真相进行验证.
- 与现有的统计和嵌入式神经网络特征选择方法进行比较.
结论:
- 提出的方法有效地估计了神经网络中排名和选择的特征意义.
- 这种方法增强了对特征重要性和模型可解释性的理解.
- 该方法的可转移性使其对分析现有和活跃的神经网络模型具有价值.
相关概念视频
Weighted Mean
4.9K
While taking the arithmetic, geometric, or harmonic mean of a sample data set, equal importance is assigned to all the data points. However, all the values may not always be equally important in some data sets. An intrinsic bias might make it more important to give more weightage to specific values over others.
For example, consider the number of goals scored in the matches of a tournament. While computing the average number of goals scored in the tournament, it may be more important to...
For example, consider the number of goals scored in the matches of a tournament. While computing the average number of goals scored in the tournament, it may be more important to...
4.9K
Gravimetry: Overview
5.0K
Gravimetric analysis is a quantitative method where the analyte is isolated and weighed directly or after conversion into a substance of known composition. Gravimetric analysis can be classified as precipitation, electrogravimetry, volatilization, and particulate gravimetry, based on the method used to isolate the analyte.
In precipitation gravimetry, the analyte is converted into a precipitate and weighed. For example, the silver content in a sample can be estimated by precipitating and...
In precipitation gravimetry, the analyte is converted into a precipitate and weighed. For example, the silver content in a sample can be estimated by precipitating and...
5.0K
Profile Leveling and Cross Sections
152
Profile leveling and cross-sections are surveying methods used to determine and document terrain elevations for infrastructure projects such as highways, railroads, canals, and pipelines. These methods provide data for earthwork planning and alignment of proposed routes. Profile leveling involves measuring elevations along a fixed line to create a vertical terrain profile. A surveyor sets up a leveling instrument at the benchmark (BM) and records a backsight (BS) to determine the...
152
What are Estimates?
5.0K
It isn't easy to measure a parameter such as the mean height or the mean weight of a population. So, we draw samples from the population and calculate the mean height or mean weight of the individuals in the sample. This sample data acts as a representative measure of the population parameter. These sample statistics are known as estimates.
The estimate for the mean of a sample is denoted by ͞x, whereas the mean of the population is designated as μ. Further, parameters such...
The estimate for the mean of a sample is denoted by ͞x, whereas the mean of the population is designated as μ. Further, parameters such...
5.0K
Design Example: Aggregate Gradation
91
The right type and quality of aggregates are crucial for concrete as they significantly influence its properties, mix proportions, and cost-effectiveness. If different sources are available for sand, the commonly used fine aggregate in concrete, the selection of sand is primarily based on its gradation.
The grading, or particle-size distribution, of sand is determined using sieve analysis, with standard sizes ranging from 150 μm to 10 mm (ASTM No. 100 sieve to 3⁄8 in. sieve). Sand is...
The grading, or particle-size distribution, of sand is determined using sieve analysis, with standard sizes ranging from 150 μm to 10 mm (ASTM No. 100 sieve to 3⁄8 in. sieve). Sand is...
91
Gradient and Del Operator
2.5K
In mathematics and physics, the gradient and del operator are fundamental concepts used to describe the behavior of functions and fields in space. The gradient is a mathematical operator that gives both the magnitude and direction of the maximum spatial rate of change. Consider a person standing on a mountain. The slope of the mountain at any given point is not defined unless it is quantified in a particular direction. For this reason, a "directional derivative" is defined, which is a vector...
2.5K


