タイムシリーズの欠落したデータインプテーション方法のベンチマークは,現実世界のテストケースを使用します
Adedolapo Aishat Toye1, Asuman Celik1, Samantha Kleinberg1
1Department of Computer Science, Stevens Institute of Technology, USA.
まとめ
ランダムな欠落したデータではなく 現実的なパターンで最適です 線形インターポレーションは,すべての欠落したデータ型において最も低い誤差を示し,複雑な欠落についてよりよい評価と帰算の技術が必要であることを強調した.
科学分野:
- 医療データサイエンス
- バイオ統計学
- 医学における機械学習
背景:
- 欠けているデータは 医療分析における大きな課題です
- 現在の計算方法は,非現実的な欠落したデータパターンで評価されることが多い.
- 現実世界の欠落メカニズム (MCAR,MAR,NMAR) は,堅固な帰算戦略を必要とします.
研究 の 目的:
- 3つの欠落したデータメカニズム (MCAR,MAR,NMAR) に関する12の割り算方法の実際の精度を評価する.
- 継続的なグルコースモニタリングと心拍数タイムシリーズのデータを比較する.
- 欠落率 (5-30%) が割り算の精度に与える影響を評価する.
主な方法:
- MCAR,MAR,NMARのメカニズムに従って,Loop (CGM) とAll of Us (心拍数) のデータセットでシミュレートされた欠落.
- 12つの最先端でよく使われる計算方法をテストした.
- ルーツ・メア・スクエア・エラー (RMSE) と 人口集団全体におけるバイアス・メトリックを用いて精度が評価された.
主要な成果:
- ランダムに欠けているデータ (MAR) とランダムに欠けているデータ (NMAR) と比較して,完全にランダムに欠けているデータ (MCAR) の推定精度は有意に高かった.
- 線形インターポレーションにより,試験されたすべてのメカニズムと人口集団において,最低のRMSEと最小のバイアスが示されました.
- 既存の評価慣行は,実世界のシナリオにおける帰算方法のパフォーマンスを過大評価する可能性があります.
結論:
- 現在の計算方法の評価は,現実的なデータパターンを欠いた現実的なパフォーマンスを反映していません.
- 線形インターポレーションは,複雑な欠損であっても,割り算のための信頼性の高いベースラインを提供します.
- 将来の研究は,現実の世界で欠けているデータメカニズムに合わせた改善された評価方法論と帰算技術の開発に焦点を当てるべきである.
関連する概念動画
Detection of Gross Error: The Q Test
6.4K
When one or more data points appear far from the rest of the data, there is a need to determine whether they are outliers and whether they should be eliminated from the data set to ensure an accurate representation of the measured value. In many cases, outliers arise from gross errors (or human errors) and do not accurately reflect the underlying phenomenon. In some cases, however, these apparent outliers reflect true phenomenological differences. In these cases, we can use statistical methods...
6.4K
Censoring Survival Data
228
Survival analysis is a statistical method used to analyze time-to-event data, often employed in fields such as medicine, engineering, and social sciences. One of the key challenges in survival analysis is dealing with incomplete data, a phenomenon known as "censoring." Censoring occurs when the event of interest (such as death, relapse, or system failure) has not occurred for some individuals by the end of the study period or is otherwise unobservable, and it might have many different...
228
Survival Tree
159
Survival trees are a non-parametric method used in survival analysis to model the relationship between a set of covariates and the time until an event of interest occurs, often referred to as the "time-to-event" or "survival time." This method is particularly useful when dealing with censored data, where the event has not occurred for some individuals by the end of the study period, or when the exact time of the event is unknown.
Building a Survival Tree
Constructing a...
Building a Survival Tree
Constructing a...
159
Quantifying and Rejecting Outliers: The Grubbs Test
2.0K
Sometimes, a data set can have a recorded numerical observation that greatly deviates from the rest of the data. Assuming that the data is normally distributed, a statistical method called the Grubbs test can be used to determine whether the observation is truly an outlier. To perform a two-tailed Grubbs test, first, calculate the absolute difference between the outlier and the mean. Then, calculate the ratio between this difference and the standard deviation of the sample. This...
2.0K
Statistical Inference Techniques in Hypothesis Testing: Parametric Versus Nonparametric Data
207
Statistical inference techniques, paramount in hypothesis testing, differentiate into two broad categories: parametric and nonparametric statistics.
Parametric statistics, as the name suggests, assumes that data follow a specific distribution, often a normal distribution. This assumption enables robust hypothesis testing and estimation. Parametric methods, like the Student's t-test or Goodness-of-fit test, are frequently employed in biostatistics due to their robustness. For instance,...
Parametric statistics, as the name suggests, assumes that data follow a specific distribution, often a normal distribution. This assumption enables robust hypothesis testing and estimation. Parametric methods, like the Student's t-test or Goodness-of-fit test, are frequently employed in biostatistics due to their robustness. For instance,...
207
Prediction Intervals
2.3K
The interval estimate of any variable is known as the prediction interval. It helps decide if a point estimate is dependable.
However, the point estimate is most likely not the exact value of the population parameter, but close to it. After calculating point estimates, we construct interval estimates, called confidence intervals or prediction intervals. This prediction interval comprises a range of values unlike the point estimate and is a better predictor of the observed sample value, y.
However, the point estimate is most likely not the exact value of the population parameter, but close to it. After calculating point estimates, we construct interval estimates, called confidence intervals or prediction intervals. This prediction interval comprises a range of values unlike the point estimate and is a better predictor of the observed sample value, y.
2.3K


