Missing value imputation in a data matrix using the regularised singular value decomposition.

Area of Science:

• Statistics
• Data Science
• Bioinformatics

Background:

• Statistical analyses often require complete data matrices, but missing data is a common challenge in database construction.
• Estimating and imputing missing data is a crucial step for maintaining data integrity and enabling robust analysis.

Purpose of the Study:

• To propose and evaluate a novel imputation method that enhances data matrix completion.
• To improve the quality of missing data imputation by incorporating regularisation into singular value decomposition (SVD).

Main Methods:

• A modified imputation technique combining regression with low-rank approximations.
• Implementation of a regularised SVD, with the regularisation parameter determined via cross-validation.
• Evaluation using ten real-world datasets from multienvironment trials with varying percentages of missing data.

Main Results:

• The regularised SVD imputation method demonstrated competitive performance against the classical approach.
• The proposed method showed superior results in several tested scenarios, effectively handling missing data.
• The inclusion of a penalised criterion in the computational algorithm smoothed eigenvectors and eigenvalues, preventing overfitting.

Conclusions:

• The regularised SVD imputation method is a robust and effective technique for addressing missing data in multivariate matrices.
• This approach offers improved imputation quality and computational stability, particularly when dealing with complex datasets.
• The method's general applicability extends to various fields requiring multivariate data analysis.