The sparse matrix transform for covariance estimation and analysis of high dimensional signals

Area of Science:

• Statistical Signal Analysis
• Machine Learning
• High-Dimensional Data Analysis

Background:

• Covariance estimation for high-dimensional signals is a challenging problem.
• Traditional methods like shrinkage and graphical lasso have limitations.

Purpose of the Study:

• To propose a novel maximum likelihood (ML) approach for covariance estimation.
• To introduce a non-linear sparsity constraint using a sparse matrix transform (SMT).
• To enable efficient and accurate covariance estimation for high-dimensional signals.

Main Methods:

• Employing a maximum likelihood (ML) framework with a novel non-linear sparsity constraint.
• Constraining the covariance to have an eigen decomposition representable as a sparse matrix transform (SMT).
• Utilizing greedy optimization of the log-likelihood function and cross-validation for parameter selection.

Main Results:

• The SMT-based covariance estimator is generally positive definite and well-conditioned, even with limited sample sizes.
• Experiments show SMT-based estimates are more accurate than shrinkage and graphical lasso estimates.
• The SMT representation allows for fast implementation of the estimated eigen-transformation.

Conclusions:

• The proposed SMT-based ML approach provides a robust and accurate method for covariance estimation in high-dimensional settings.
• The SMT offers a generalization of the Fast Fourier Transform (FFT) for fast eigen-signal analysis of non-stationary signals.
• This method enhances statistical signal analysis and machine learning applications dealing with complex data.