一种新的重要性得分基于变量选择方法和使用MIR和NIR数据集的验证
Li Jun Tang1, Xin Kang Li1, Yue Huang1
1School of Pharmacy and Food Engineering, Wuyi University, Jiangmen 529020, PR China.
Spectrochimica acta. Part A, Molecular and biomolecular spectroscopy
|January 10, 2025
概括
一种新的变量选择方法,VMHBSC,提高了光谱分析的准确性. 这种新的过程有效地识别了关键变量,改善了中红外和近红外光谱数据集的模型性能.
科学领域:
- 频谱分析是一种分析.
- 化学测量 化学测量 化学测量
- 机器学习是机器学习.
背景情况:
- 变量选择对于准确的光谱数据解释至关重要.
- 现有的方法可能无法最佳地识别最有信息的光谱变量.
研究的目的:
- 介绍一个名为VMHBSC的新的六步变量选择过程.
- 证明VMHBSC在提高光谱分析模型性能方面的有效性.
- 在中红外 (MIR) 和近红外 (NIR) 频谱数据集上评估VMHBSC.
主要方法:
- 开发并应用了VMHBSC变量选择过程.
- 使用决策树 (DT),梯度提升决策树 (GBDT) 和极端梯度提升 (XGBoost) 进行模型构建.
- 在Chenpi样本的MIR数据集和模拟竞赛的NIR数据集上测试了VMHBSC.
主要成果:
- VMHBSC从MIR数据集中的7468中确定了3个关键变量,显著提高了DT,GBDT和XGBoost模型的准确性.
- 在NIR数据集中的256个中,VMHBSC选择了24个重要的变量.
- 混合模型 (VMHBSC-DT,VMHBSC-GBDT,VMHBSC-XGBoost) 使用所选的NIR变量显示稳定的性能.
结论:
- VMHBSC过程有效地提高了光谱分析中的模型性能和稳定性.
- 在复杂的光谱数据集中,VMHBSC提供了一种强大的变量选择方法.
- 这种方法提高了化学测量模型的解释性和准确性.
相关概念视频
Critical Region, Critical Values and Significance Level
11.8K
The critical region, critical value, and significance level are interdependent concepts crucial in hypothesis testing.
In hypothesis testing, a sample statistic is converted to a test statistic using z, t, or chi-square distribution. A critical region is an area under the curve in probability distributions demarcated by the critical value. When the test statistic falls in this region, it suggests that the null hypothesis must be rejected. As this region contains all those values of the...
In hypothesis testing, a sample statistic is converted to a test statistic using z, t, or chi-square distribution. A critical region is an area under the curve in probability distributions demarcated by the critical value. When the test statistic falls in this region, it suggests that the null hypothesis must be rejected. As this region contains all those values of the...
11.8K
Significance Testing: Overview
3.3K
Significance testing is a set of statistical methods used to test whether a claim about a parameter is valid. In analytical chemistry, significance testing is used primarily to determine whether the difference between two values comes from determinate or random errors. The effect of a particular change in the measurement protocol, analyst, or sample itself can cause a deviation from the expected result. In the case of a suspected deviation/outlier, we need to be able to confirm mathematically...
3.3K
Reliability and Validity
12.7K
Reliability and validity are two important considerations that must be made with any type of data collection. Reliability refers to the ability to consistently produce a given result. In the context of psychological research, this would mean that any instruments or tools used to collect data do so in consistent, reproducible ways.
12.7K
Quantifying and Rejecting Outliers: The Grubbs Test
1.5K
Sometimes, a data set can have a recorded numerical observation that greatly deviates from the rest of the data. Assuming that the data is normally distributed, a statistical method called the Grubbs test can be used to determine whether the observation is truly an outlier. To perform a two-tailed Grubbs test, first, calculate the absolute difference between the outlier and the mean. Then, calculate the ratio between this difference and the standard deviation of the sample. This...
1.5K
Sensitivity, Specificity, and Predicted Value
180
In healthcare diagnostics, laboratory tests play a crucial role in identifying and diagnosing a wide range of medical conditions. However, interpreting test results is not always straightforward. An abnormal test result does not always confirm the presence of a disease, just as a normal result does not guarantee its absence. To assess the reliability of these diagnostic tools, healthcare practitioners rely on two key statistical indicators: sensitivity and specificity.
Sensitivity is the...
Sensitivity is the...
180
Applications of IR Spectroscopy: Overview
477
The non-destructive nature and ability to provide valuable chemical information make IR spectroscopy a versatile technique with broad applications in various scientific and industrial fields. IR spectroscopy is commonly used to identify and characterize organic and inorganic compounds. It provides information about the functional groups present in a molecule and the bonding between atoms. This helps in the structural elucidation of compounds during organic synthesis, pharmaceutical research,...
477


