Accelerated linear algebra for large scale DFT calculations of materials on CPU/GPU architectures with CRYSTAL

We discuss the implementation strategy, numerical accuracy, and computational performance of the acceleration of linear algebra operations through graphics processing units (GPUs) for the self-consistent field driver of the Crystal electronic structure package for solid state density functional theory simulations. Accelerated tasks include matrix multiplication, diagonalization, and inversion, as well as Cholesky decomposition. The scaling of the implemented strategy over multiple accelerating devices is assessed in the range of 1-8 GPUs per node and found to be remarkably regular. Tests are performed on three systems: α-quartz, a microporous zeolitic imidazolate framework (ZIF-8), and a giant mesoporous metal-organic framework (bio-MOF). Scaling with system size is investigated via supercells of increasing size of both α-quartz and ZIF-8 (up to 648 and 2208 atoms per cell, respectively). The bio-MOF model structure has 2808 atoms per cell, with 33 672 basis functions. We test the performance of the accelerated code with both generalized gradient approximation (GGA) and hybrid GGA exchange-correlation functionals. The efficiency of the new accelerated code is compared to the previous central processing unit (CPU)-only parallelization strategies based on MPI or MPI/OpenMP within either replicated or distributed memory (i.e., massively parallel) approaches. Such a comparison highlights how the new GPU-accelerated code enables calculations on large systems at a significantly reduced computational cost relative to CPU-only strategies. For instance, we find that for the bio-MOF system, the computing time of the linear algebra tasks from a single GPU is comparable to that from the reference approach in the range of 512-1024 CPU cores and 4-8 nodes.