Acceleration of Approximate Matrix Multiplications on GPUs

Area of Science:

• Scientific Computing
• Numerical Analysis
• High-Performance Computing

Background:

• Matrix multiplication is crucial for scientific computing, impacting eigenvalue computation and optimization.
• Existing methods like GPUs and specialized libraries accelerate matrix products, but accelerating approximate matrix multiplication (AMM) on GPUs remains underexplored.

Purpose of the Study:

• To develop an enhanced Monte Carlo AMM method optimized for GPU acceleration.
• To provide an analytical solution for optimal hyperparameter tuning in the proposed AMM method.
• To demonstrate the method's effectiveness in accelerating eigenvalue computations.

Main Methods:

• Proposed a novel Monte Carlo AMM algorithm designed for parallel GPU execution.
• Derived an analytical solution for optimizing hyperparameters within the AMM method.
• Integrated the enhanced AMM into the power method for eigenvalue calculation.

Main Results:

• The proposed AMM method enhances approximation accuracy without increasing computation time compared to conventional AMMs.
• The method is well-suited for GPU parallelization and integration into diverse algorithms.
• Application to eigenvalue computation on an NVIDIA A100 GPU halved the computation time versus the standard power method using cuBLAS.

Conclusions:

• The novel AMM method offers significant speedups for scientific computations on GPUs.
• This research bridges the gap in accelerating AMMs for general matrices on GPUs.
• The method shows promise for broader applications in scientific and practical calculations requiring fast matrix products.