Related Experiment Video
Updated: Mar 13, 2026
12:18
A Machine Learning Approach to Design an Efficient Selective Screening of Mild Cognitive Impairment
Published on: January 11, 2020
8.2K
Data Driven Estimation of Imputation Error-A Strategy for Imputation with a Reject Option.
1Center for Neuropsychiatric Schizophrenia Research (CNSR) & Center for Clinical Intervention and Neuropsychiatric Schizophrenia Research (CINS), Psychiatric Center Glostrup, Copenhagen University Hospitals, Mental Health Services, Capital Region of Denmark, Glostrup, Denmark.
Plos One
|October 11, 2016
Summary
This study introduces a machine learning method to estimate imputation error for missing data, helping researchers decide which cases to impute. This approach improves data analysis by providing informed choices for handling missing values.
Area of Science:
- Statistics
- Data Science
- Machine Learning
Background:
- Missing data is a pervasive challenge across research disciplines.
- Current imputation methods often apply indiscriminately, ignoring the impact of specific missing data patterns.
- The effects of imputation can vary significantly depending on the nature of the missing information.
Purpose of the Study:
- To propose a machine learning approach for estimating imputation error on a per-case basis.
- To provide researchers with a practical tool for making informed decisions about data imputation.
- To enable the evaluation of different imputation method performances.
Main Methods:
- Simulating all missing value patterns within complete datasets to calculate the 'true error'.
- Estimating imputation error for cases with missing data by weighting 'true errors' based on similarity.
- Utilizing complete cases to assess the impact of error thresholds.
Main Results:
- The proposed method quantifies imputation error for individual cases with missing data.
- It offers a practical way to guide decisions on which records benefit most from imputation.
- The approach facilitates the testing and comparison of various imputation techniques.
Conclusions:
- The developed machine learning method provides a nuanced approach to handling missing data through informed imputation.
- Researchers can use this tool to select appropriate imputation strategies based on estimated error.
- The method supports user-defined error thresholds, promoting transparency and data-driven decision-making over conventional practices.
Related Concept Videos
01:00Detection of Gross Error: The Q Test
7.2K
When one or more data points appear far from the rest of the data, there is a need to determine whether they are outliers and whether they should be eliminated from the data set to ensure an accurate representation of the measured value. In many cases, outliers arise from gross errors (or human errors) and do not accurately reflect the underlying phenomenon. In some cases, however, these apparent outliers reflect true phenomenological differences. In these cases, we can use statistical methods...
7.2K
01:14Errors In Hypothesis Tests
6.2K
When performing a hypothesis test, there are four possible outcomes depending on the actual truth (or falseness) of the null hypothesis and the decision to reject or not.
6.2K
01:02Quantifying and Rejecting Outliers: The Grubbs Test
4.4K
Sometimes, a data set can have a recorded numerical observation that greatly deviates from the rest of the data. Assuming that the data is normally distributed, a statistical method called the Grubbs test can be used to determine whether the observation is truly an outlier. To perform a two-tailed Grubbs test, first, calculate the absolute difference between the outlier and the mean. Then, calculate the ratio between this difference and the standard deviation of the sample. This...
4.4K
01:13Accuracy and Errors in Hypothesis Testing
661
Hypothesis testing is a fundamental statistical tool that begins with the assumption that the null hypothesis H0 is true. During this process, two types of errors can occur: Type I and Type II. A Type I error refers to the incorrect rejection of a true null hypothesis, while a Type II error involves the failure to reject a false null hypothesis.
In hypothesis testing, the probability of making a Type I error, denoted as α, is commonly set at 0.05. This significance level indicates a 5%...
In hypothesis testing, the probability of making a Type I error, denoted as α, is commonly set at 0.05. This significance level indicates a 5%...
661
01:09Decision Making: P-value Method
7.1K
The process of hypothesis testing based on the P-value method includes calculating the P- value using the sample data and interpreting it.
First, a specific claim about the population parameter is proposed. The claim is based on the research question and is stated in a simple form. Further, an opposing statement to the claim is also stated. These statements can act as null and alternative hypotheses: a null hypothesis would be a neutral statement while the alternative hypothesis can...
First, a specific claim about the population parameter is proposed. The claim is based on the research question and is stated in a simple form. Further, an opposing statement to the claim is also stated. These statements can act as null and alternative hypotheses: a null hypothesis would be a neutral statement while the alternative hypothesis can...
7.1K
01:04Random Error
10.0K
Random or indeterminate errors originate from various uncontrollable variables, such as variations in environmental conditions, instrument imperfections, or the inherent variability of the phenomena being measured. Usually, these errors cannot be predicted, estimated, or characterized because their direction and magnitude often vary in magnitude and direction even during consecutive measurements. As a result, they are difficult to eliminate. However, the aggregate effect of these errors can be...
10.0K