Factor analysis models via I-divergence optimization.

Area of Science:

• Statistics
• Machine Learning
• Covariance Matrix Approximation

Background:

• Covariance matrices are fundamental in statistical modeling.
• Approximating large covariance matrices is crucial for computational efficiency.
• Existing methods like EM algorithm have limitations in factor analysis.

Purpose of the Study:

• To develop a novel algorithm for approximating positive definite covariance matrices.
• To minimize the I-divergence between normal distributions with different covariance structures.
• To compare the performance of the new algorithm against existing methods.

Main Methods:

• Constructed an iterative alternating minimization algorithm (AML) based on Csiszár-Tusnády.
• Employed a lifted space for optimization to determine the optimal (H, D) pair.
• Developed a variant, ACML, inspired by ECME, for improved performance.
• Analyzed algorithm convergence, including cases with singular D.

Main Results:

• The proposed AML algorithm effectively approximates covariance matrices.
• The choice of the enlarged space is critical for AML optimization.
• ACML demonstrates competitive performance against the EM algorithm in numerical experiments.
• Convergence properties of AML were theoretically studied.

Conclusions:

• The AML and ACML algorithms offer efficient alternatives for covariance matrix approximation.
• These methods are valuable for applications in statistical modeling and machine learning.
• Further research can explore extensions and applications of these algorithms.