Multi-GPU Immersed Boundary Method Hemodynamics Simulations.

Area of Science:

• Computational fluid dynamics
• Biomedical engineering
• High-performance computing

Background:

• Cell-resolved blood flow simulations are gaining traction due to algorithmic and computational advancements.
• Fluid structure interaction (FSI) methods, particularly those using the immersed boundary method, are common for modeling cell-resolved hemodynamics.
• Graphics Processing Units (GPUs) offer acceleration but introduce communication challenges in heterogeneous CPU-GPU systems.

Purpose of the Study:

• To conduct and analyze the largest distributed GPU-accelerated FSI simulations of high hematocrit, cell-resolved blood flow.
• To compare the performance and scalability of these simulations on different GPU configurations (fat nodes vs. single GPU per node).
• To identify performance bottlenecks in multiscale multi-grid FSI simulations on heterogeneous computing architectures.

Main Methods:

• Implementation of distributed GPU-accelerated fluid structure interaction (FSI) algorithms.
• Simulation of blood flow with over 17 million individual red blood cell models.
• Comparative performance analysis across heterogeneous systems with varying GPU densities (e.g., 6 GPUs/node vs. 1 GPU/node).
• Benchmarking of CPU-based versus GPU-based implementations to quantify data movement costs.

Main Results:

• Successfully executed the largest distributed GPU-accelerated FSI simulations for high hematocrit blood flow to date.
• Identified data movement between CPU and GPU as the most significant performance bottleneck in multiscale multi-grid FSI simulations on heterogeneous systems.
• Demonstrated performance differences based on GPU node architecture.

Conclusions:

• GPU acceleration is feasible for large-scale cell-resolved blood flow simulations.
• Optimizing data transfer strategies is crucial for maximizing performance and scalability on heterogeneous CPU-GPU systems.
• Further research is needed to mitigate data movement bottlenecks for exascale computing.