基于PCA-DBSCAN的异常值去除方法用于血液-SERS数据分析.
Miaomiao Liu1, Tingyin Wang1, Qiyi Zhang1
1Key Laboratory of OptoElectronic Science and Technology for Medicine of Ministry of Education, Fujian Provincial Key Laboratory of Photonics Technology, Fujian Normal University, Fuzhou, 350117, China. tywang@fjnu.edu.cn.
Analytical methods : advancing methods and applications
|January 17, 2024
概括
一种新的方法,主要组件分析和基于密度的应用程序与噪音的空间聚类 (PCA-DBSCAN),有效地从癌症查数据中删除异常值. 这提高了用于早期癌症检测的表面增强拉曼光谱 (SERS) 模型的准确性.
科学领域:
- 频谱学是一种光谱学.
- 数据科学数据科学数据科学
- 生物医学工程 生物医学工程
背景情况:
- 表面增强拉曼光谱 (SERS) 显示了癌症查的潜力.
- 拉曼光谱可能会受到环境因素和样本降解的影响,导致异常值.
- 现有的方法往往忽略了分类数据中的异常值,影响了模型性能.
研究的目的:
- 为SERS癌症查数据引入一种新的异常值去除方法.
- 提高基于SERS的癌症检测模型的准确性和可靠性.
- 证明拟议方法在各种科学领域的多功能性.
主要方法:
- 拟议的主要组件分析和基于密度的空间集群应用与噪声 (PCA-DBSCAN) 异常检测.
- 利用缩小维度和光谱数据聚类来识别和删除异常值.
- 优化PCA-DBSCAN参数 (Eps,MinPts) 和用于癌症查的机器学习模型.
主要成果:
- 该PCA-DBSCAN方法有效地识别并从SERS数据集中删除异常值.
- 移除异常值显著提高了癌症查模型的性能.
- 在优化模型中实现了97.41%的宏观平均召回率和97.74%的宏观平均F1得分.
结论:
- PCA-DBSCAN是SERS数据中异常值去除的有效方法,提高了癌症查的准确性.
- 拟议的方法比以往基于SERS的癌症检测方法有显著的改进.
- PCA-DBSCAN技术在各种研究和工业领域的异常值检测中具有广泛的适用性.
相关概念视频
Quantifying and Rejecting Outliers: The Grubbs Test
1.6K
Sometimes, a data set can have a recorded numerical observation that greatly deviates from the rest of the data. Assuming that the data is normally distributed, a statistical method called the Grubbs test can be used to determine whether the observation is truly an outlier. To perform a two-tailed Grubbs test, first, calculate the absolute difference between the outlier and the mean. Then, calculate the ratio between this difference and the standard deviation of the sample. This...
1.6K
Detection of Gross Error: The Q Test
6.1K
When one or more data points appear far from the rest of the data, there is a need to determine whether they are outliers and whether they should be eliminated from the data set to ensure an accurate representation of the measured value. In many cases, outliers arise from gross errors (or human errors) and do not accurately reflect the underlying phenomenon. In some cases, however, these apparent outliers reflect true phenomenological differences. In these cases, we can use statistical methods...
6.1K
What Are Outliers?
3.9K
Outliers are observed data points that are far from the least squares line. They have unusual values and need to be examined carefully. Though an outlier may result from erroneous data, at other times, it may hold valuable information about the population under study and should be included in the data. Hence, it is crucial to examine what causes a data point to be an outlier.
The z score is used to find outliers or unusual values. It should be noted that any values beyond -2 and +2 are...
The z score is used to find outliers or unusual values. It should be noted that any values beyond -2 and +2 are...
3.9K
Outliers and Influential Points
4.1K
An outlier is an observation of data that does not fit the rest of the data. It is sometimes called an extreme value. When you graph an outlier, it will appear not to fit the pattern of the graph. Some outliers are due to mistakes (for example, writing down 50 instead of 500), while others may indicate that something unusual is happening. Outliers are present far from the least squares line in the vertical direction. They have large "errors," where the "error" or residual is the...
4.1K
Modified Boxplots
9.7K
A standard box and whisker plot informs us about the spread of the data in a given sample. One can identify the minimum value, maximum value, first quartile value, second quartile or median value, and third quartile.
However, the box plot does not tell the reader about outliers - values that lie far from the center of the data. We can modify the standard box and whisker plot to identify the outliers and visualize the actual spread of the data in a sample.
Initially, we calculate the adjusted...
However, the box plot does not tell the reader about outliers - values that lie far from the center of the data. We can modify the standard box and whisker plot to identify the outliers and visualize the actual spread of the data in a sample.
Initially, we calculate the adjusted...
9.7K


