Extraordinarily Time- and Memory-Efficient Large-Scale Canonical Correlation Analysis in Fourier Domain: From Shallow...

Area of Science:

• Machine Learning
• Statistics
• Signal Processing

Background:

• Canonical Correlation Analysis (CCA) is a vital statistical technique.
• Traditional CCA faces computational challenges with large datasets, limiting its practical application.
• High complexity burdens processing units and memory, making large-scale CCA nearly impractical.

Purpose of the Study:

• To develop a novel, efficient CCA method for large-scale datasets.
• To overcome the computational and memory limitations of conventional CCA.
• To adapt CCA for nonlinear kernel and deep learning models.

Main Methods:

• Developed a CCA method operating in the Fourier domain.
• Transformed eigenvector computation into learning discriminative Fourier bases via element-wise operations.
• Implemented a progressive estimation scheme for eigenvalues using partial samples and batch processing.
• Extended the method to nonlinear kernel and deep learning models.

Main Results:

• The proposed Fourier domain CCA (FFT-CCA) demonstrates extraordinary speed and memory efficiency.
• Achieved comparable accuracy to state-of-the-art methods but with training times up to 1000 times faster.
• Validated on large-scale datasets like MNIST8M, X-RAY MICROBEAM SPEECH, and Twitter Users Data.
• Demonstrated satisfactory accuracy and extremely fast training for nonlinear and deep models.

Conclusions:

• The FFT-CCA method offers a significant advancement for large-scale correlation analysis.
• This approach makes CCA a practical and efficient tool for big data applications.
• The proposed models represent best practices for handling large-scale correlation datasets.