Hybrid Message-Passing Interface-Open Multiprocessing Accelerated Euler-Lagrange Simulations of Microbubble Enhanced...

Microbubble enhanced high intensity focused ultrasound (HIFU) is of great interest to tissue ablation for solid tumor treatments such as in liver and brain cancers, in which contrast agents/microbubbles are injected into the targeted region to promote heating and reduce prefocal tissue damage. A compressible Euler-Lagrange coupled model has been developed to accurately characterize the acoustic and thermal fields during this process. This employs a compressible Navier-Stokes solver for the ultrasound acoustic field and a discrete singularities model for bubble dynamics. To address the demanding computational cost relevant to practical medical applications, a multilevel hybrid message-passing interface (MPI)-open multiprocessing (OpenMP) parallelization scheme is developed to take advantage of both scalability of MPI and load balancing of OpenMP. At the first level, the Eulerian computational domain is divided into multiple subdomains and the bubbles are subdivided into groups based on which subdomain they fall into. At the next level, in each subdomain containing bubbles, multiple OpenMP threads are activated to speed up the computations of the bubble dynamics. For improved throughput, the OpenMP threads are more heavily distributed to subdomains where the bubbles are clustered. By doing this, MPI load imbalance issue due to uneven bubble distribution is mitigated by OpenMP speedup locally for those subdomains hosting more bubbles than others. The hybrid MPI-OpenMP Euler-Lagrange solver is used to conduct simulations and physical studies of bubble-enhanced HIFU problems containing a large number of microbubbles. The phenomenon of acoustic shadowing caused by the bubble cloud is then analyzed and discussed. Efficiency tests on two different machines with 48 processors are conducted and indicate 2-3 times speedup with the same hardware by introducing an OpenMP parallelization in combination with the MPI parallelization.