A Benchmark for Data Imputation Methods.

Area of Science:

• Data Science
• Machine Learning
• Data Engineering

Background:

• Data quality is a critical challenge in modern applications, especially for machine learning (ML).
• Missing values are a frequent data quality issue that can disrupt data pipelines and negatively impact ML model performance.
• Existing imputation methods lack comprehensive, fair benchmarking under realistic conditions.

Purpose of the Study:

• To bridge the gap in benchmarking data imputation techniques.
• To compare classical and novel deep learning imputation methods.
• To evaluate imputation performance on both imputation quality and downstream ML task impact.

Main Methods:

• Conducted extensive experiments on diverse datasets with realistic missing data patterns.
• Compared classical ML imputation methods with novel deep learning approaches.
• Assessed methods under scenarios where missing data affects only test or both training and test sets.

Main Results:

• Provided valuable insights into the performance of various imputation methods.
• Highlighted the impact of different imputation strategies on downstream ML tasks.
• Identified effective imputation techniques under realistic data conditions.

Conclusions:

• The study offers guidance for researchers and engineers in selecting data preprocessing methods.
• Results aid in automated data quality improvement for robust ML applications.
• Informed selection of imputation techniques can enhance predictive performance and responsible AI.