TRANSPOSABLE REGULARIZED COVARIANCE MODELS WITH AN APPLICATION TO MISSING DATA IMPUTATION.

Area of Science:

• Statistics
• Machine Learning
• Data Science

Background:

• Missing data estimation is a significant challenge in high-dimensional datasets structured as matrices.
• Transposable data allows rows, columns, or both to be treated as features, requiring specialized modeling techniques.

Purpose of the Study:

• To develop a novel statistical framework for modeling and imputing missing data in transposable, high-dimensional matrices.
• To introduce the mean-restricted matrix-variate normal distribution and transposable regularized covariance models.

Main Methods:

• Modification of the matrix-variate normal distribution to the mean-restricted matrix-variate normal.
• Application of additive penalties on inverse covariance matrices for maximum likelihood estimation.
• Formulation of EM-type algorithms for missing data imputation in multivariate and transposable settings.

Main Results:

• Theoretical results demonstrate the applicability of transposable models to high-dimensional data.
• Simulations and real-world data analysis (microarray, Netflix) show improved imputation performance over existing methods.
• The proposed techniques offer enhanced flexibility in handling complex data structures.

Conclusions:

• The developed mean-restricted matrix-variate normal models and imputation algorithms effectively address missing data in high-dimensional transposable matrices.
• These methods provide a flexible and often superior alternative to current imputation techniques for complex datasets.