Missing value imputation using least squares techniques in contaminated matrices.

Area of Science:

• Data Science
• Statistical Modeling
• Matrix Approximation

Background:

• Matrix imputation methods using least squares can be sensitive to outliers.
• Outliers can degrade imputation quality and cause convergence issues.

Purpose of the Study:

• To develop and evaluate pre-processing strategies for robust data imputation.
• To mitigate the impact of outliers on matrix imputation algorithms.

Main Methods:

• Explored pre-processing options before applying a mixture of regression and lower rank approximation.
• Investigated robust singular value decomposition (SVD) and outlier detection followed by treating outliers as missing data.
• Evaluated methods using cross-validation on real-world multi-environment trial data.

Main Results:

• Proposed pre-processing methods significantly improve imputation quality compared to the original algorithm.
• Robust SVD effectively handles outliers and enhances the imputation procedure.
• Outlier detection and treatment as missing data also yielded robust imputation results.

Conclusions:

• The original imputation method is susceptible to outliers and should be replaced by robust alternatives.
• Pre-processing steps are crucial for reliable matrix imputation, especially with suspected data contamination.
• The proposed methods ensure algorithm performance on datasets with potential outlier contamination.