Massively scalable workflows for quantum chemistry: BigChem and ChemCloud

Area of Science:

• Computational Chemistry
• Quantum Chemistry
• Distributed Computing

Background:

• Electronic structure theory is foundational for computational chemistry.
• Existing methods often face limitations in scalability and resource utilization for large-scale problems.

Purpose of the Study:

• Introduce BigChem, a novel distributed computing framework for quantum chemistry.
• Enable efficient parallel computation of complex quantum chemistry problems.
• Provide a scalable and fault-tolerant platform for computational chemistry research.

Main Methods:

• Developed a composable framework leveraging industry-standard middleware (Celery, RabbitMQ, Redis).
• Integrated with MolSSI packages like QCEngine for standardized execution of computational chemistry programs (Psi4, xtb, geomeTRIC, TeraChem).
• Designed for heterogeneous computing environments including cloud, local clusters, and supercomputers.

Main Results:

• Demonstrated full utilization of compute resources at scale.
• Achieved linear scalability with BigChem on up to 125 GPUs for computational chemistry workloads.
• Introduced ChemCloud, a web API for scalable and secure access to BigChem.

Conclusions:

• BigChem provides a robust and scalable solution for demanding quantum chemistry calculations.
• The framework facilitates the design of sophisticated quantum chemistry workflows.
• ChemCloud enhances accessibility to advanced computational chemistry resources.