Coupled singular value decomposition of a cross-covariance matrix

Area of Science:

• Machine Learning
• Linear Algebra
• Signal Processing

Background:

• Singular Value Decomposition (SVD) is a fundamental matrix factorization technique.
• On-line learning algorithms update models incrementally as new data arrives.
• Estimating singular values and vectors simultaneously in an on-line setting presents challenges.

Purpose of the Study:

• To develop coupled on-line learning rules for SVD of cross-covariance matrices.
• To integrate singular value estimation directly into the learning rules for singular vectors.
• To enhance decorrelation methods for multi-dimensional SVD estimation.

Main Methods:

• Derivation of coupled on-line learning rules for SVD.
• Incorporation of singular value estimates to modulate learning rates for singular vectors.
• Application of a first-order Gram-Schmidt orthonormalization approximation for decorrelation.
• Experimental validation using synthetic datasets.

Main Results:

• Coupled learning rules demonstrate faster convergence compared to Hebbian learning rules.
• The first-order Gram-Schmidt approximation yields more precise SVD estimates.
• The proposed method achieves superior orthonormality of estimated singular vectors over standard deflation techniques.

Conclusions:

• Coupled on-line SVD learning rules provide an efficient and accurate approach for matrix decomposition.
• The novel decorrelation method enhances the performance of on-line SVD algorithms.
• This work advances the capabilities of adaptive signal processing and machine learning algorithms.