A survey on missing data in machine learning.

Area of Science:

• Data Science
• Machine Learning
• Statistics

Background:

• Missing values are a common challenge in data analysis, arising from various factors and potentially biasing results.
• Ignoring missing data can lead to inaccurate conclusions in machine learning models.
• Effective imputation techniques are essential for robust data pre-processing.

Purpose of the Study:

• To review and aggregate literature on machine learning techniques for handling missing data.
• To provide insights into the performance, limitations, and suitability of different imputation methods.
• To propose and evaluate two specific imputation methods: k-nearest neighbor and missForest.

Main Methods:

• Literature review focusing on machine learning-based missing data imputation.
• Implementation and evaluation of the k-nearest neighbor imputation algorithm.
• Implementation and evaluation of the missForest algorithm, an iterative imputation method based on random forests.

Main Results:

• Both k-nearest neighbor and missForest demonstrated successful handling of missing values in the evaluated datasets.
• Performance was assessed on the Iris dataset and a novel power plant fan dataset with induced missingness (5-20%).
• The study highlights the practical applicability of these machine learning imputation techniques.

Conclusions:

• Machine learning imputation methods, specifically k-nearest neighbor and missForest, are effective in addressing missing data challenges.
• These methods offer viable solutions to prevent biased analysis caused by missing values.
• Further research directions in missing data imputation using machine learning are suggested.