使用现实世界的测试案例对时间序列的缺失数据归算方法进行基准测试
Adedolapo Aishat Toye1, Asuman Celik1, Samantha Kleinberg1
1Department of Computer Science, Stevens Institute of Technology, USA.
概括
在随机缺失的数据,而不是现实的模式上运行最好. 线性插入显示所有缺失数据类型的误差最低,强调需要更好的复杂缺失的评估和归算技术.
科学领域:
- 医疗数据科学
- 生物统计学
- 医学中的机器学习
背景情况:
- 缺少数据是医疗分析的一个重大挑战.
- 目前的归算方法通常是基于不切实际的缺失数据模式进行评估的.
- 现实世界缺失机制 (MCAR,MAR,NMAR) 需要强大的归算策略.
研究的目的:
- 通过三种缺失数据机制 (MCAR,MAR,NMAR) 评估12种归算方法的真实准确性.
- 为了比较连续血糖监测和心率时间序列数据的归算性能.
- 评估缺失率 (5-30%) 对归算准确性的影响.
主要方法:
- 根据MCAR,MAR和NMAR机制在Loop (CGM) 和All of Us (心率) 数据集中的模拟失踪.
- 测试了12种最先进和常用的归算方法.
- 在人口群体中使用根平均平方误差 (RMSE) 和偏差指标评估准确性.
主要成果:
- 与随机缺失 (MAR) 和非随机缺失 (NMAR) 数据相比,完全随机缺失 (MCAR) 数据的归算精度显著提高.
- 在所有测试的机制和人口群体中,线性插入显示了最低的RMSE和最小的偏差.
- 现有的评估实践可能会在现实场景中高估归算方法的性能.
结论:
- 目前的归算方法评估不反映现实世界的表现,缺乏现实的数据模式.
- 线性插入提供了可靠的归算基线,即使是复杂的缺失.
- 进一步的研究应该集中在开发改进的评估方法和归算技术,以适应现实世界缺失数据机制.
相关概念视频
Detection of Gross Error: The Q Test
6.4K
When one or more data points appear far from the rest of the data, there is a need to determine whether they are outliers and whether they should be eliminated from the data set to ensure an accurate representation of the measured value. In many cases, outliers arise from gross errors (or human errors) and do not accurately reflect the underlying phenomenon. In some cases, however, these apparent outliers reflect true phenomenological differences. In these cases, we can use statistical methods...
6.4K
Censoring Survival Data
228
Survival analysis is a statistical method used to analyze time-to-event data, often employed in fields such as medicine, engineering, and social sciences. One of the key challenges in survival analysis is dealing with incomplete data, a phenomenon known as "censoring." Censoring occurs when the event of interest (such as death, relapse, or system failure) has not occurred for some individuals by the end of the study period or is otherwise unobservable, and it might have many different...
228
Survival Tree
159
Survival trees are a non-parametric method used in survival analysis to model the relationship between a set of covariates and the time until an event of interest occurs, often referred to as the "time-to-event" or "survival time." This method is particularly useful when dealing with censored data, where the event has not occurred for some individuals by the end of the study period, or when the exact time of the event is unknown.
Building a Survival Tree
Constructing a...
Building a Survival Tree
Constructing a...
159
Quantifying and Rejecting Outliers: The Grubbs Test
2.0K
Sometimes, a data set can have a recorded numerical observation that greatly deviates from the rest of the data. Assuming that the data is normally distributed, a statistical method called the Grubbs test can be used to determine whether the observation is truly an outlier. To perform a two-tailed Grubbs test, first, calculate the absolute difference between the outlier and the mean. Then, calculate the ratio between this difference and the standard deviation of the sample. This...
2.0K
Statistical Inference Techniques in Hypothesis Testing: Parametric Versus Nonparametric Data
207
Statistical inference techniques, paramount in hypothesis testing, differentiate into two broad categories: parametric and nonparametric statistics.
Parametric statistics, as the name suggests, assumes that data follow a specific distribution, often a normal distribution. This assumption enables robust hypothesis testing and estimation. Parametric methods, like the Student's t-test or Goodness-of-fit test, are frequently employed in biostatistics due to their robustness. For instance,...
Parametric statistics, as the name suggests, assumes that data follow a specific distribution, often a normal distribution. This assumption enables robust hypothesis testing and estimation. Parametric methods, like the Student's t-test or Goodness-of-fit test, are frequently employed in biostatistics due to their robustness. For instance,...
207
Prediction Intervals
2.3K
The interval estimate of any variable is known as the prediction interval. It helps decide if a point estimate is dependable.
However, the point estimate is most likely not the exact value of the population parameter, but close to it. After calculating point estimates, we construct interval estimates, called confidence intervals or prediction intervals. This prediction interval comprises a range of values unlike the point estimate and is a better predictor of the observed sample value, y.
However, the point estimate is most likely not the exact value of the population parameter, but close to it. After calculating point estimates, we construct interval estimates, called confidence intervals or prediction intervals. This prediction interval comprises a range of values unlike the point estimate and is a better predictor of the observed sample value, y.
2.3K


