Graphics Processing Units and High-Dimensional Optimization.

Area of Science:

• Computational Statistics
• High-Dimensional Optimization
• Parallel Computing

Background:

• Graphics Processing Units (GPUs) offer massive parallel processing capabilities.
• Traditional statistical algorithms can be computationally intensive, especially in high dimensions.
• Exploiting GPU power requires optimization algorithms that can be parallelized.

Purpose of the Study:

• To explore the potential of GPUs for accelerating high-dimensional optimization problems.
• To identify optimization algorithms suitable for GPU implementation.
• To demonstrate the practical utility of GPUs in statistical applications.

Main Methods:

• Identifying optimization algorithms that decompose into parallel tasks with limited data access.
• Focusing on Expectation-Maximization (EM) and Majorization-Minimization (MM) algorithms.
• Applying GPU acceleration to nonnegative matrix factorization, PET image reconstruction, and multidimensional scaling.

Main Results:

• GPUs can dramatically accelerate many statistical algorithms.
• Algorithms like EM and MM are well-suited for GPU parallelization.
• Speedups of up to 100-fold were achieved in demonstrated applications.

Conclusions:

• GPUs are poised to revolutionize computational statistics.
• Statisticians should adopt GPU computing for high-dimensional problems.
• GPU acceleration offers significant performance gains for complex statistical tasks.