Optimizing High-Performance Computing Systems for Biomedical Workloads

Area of Science:

• Computational Biology
• High-Performance Computing (HPC)
• Bioinformatics

Background:

• Computational biologist productivity is often limited by slow workflows and non-optimized computing systems.
• Biomedical researchers prioritize scientific discovery over code optimization, hindering progress.
• Many life science applications underutilize HPC capabilities, necessitating system tuning.

Purpose of the Study:

• To present a case study on optimizing HPC systems for biomedical computational workloads.
• To demonstrate strategies for enhancing scientific throughput and data fidelity.
• To share practical approaches for sustainable HPC system growth and management.

Main Methods:

• Analyzed the impact of computational applications on system components (cores, file system, resource manager).
• Iteratively upgraded HPC system architecture, including scheduling, memory, compute, and file system capabilities.
• Implemented a chargeback fee structure for long-term system stability and sustainability.

Main Results:

• Evolved a 70 teraflop system to a 1.4 petaflop system over seven years, supporting a nearly 10-fold user base growth.
• Enabled over 900 biomedical publications across genetics, gene expression, machine learning, and structural/chemical biology.
• Maintained minimal impact on existing user workflows and code during system optimization.

Conclusions:

• Effective HPC system stewardship requires understanding application-system interactions for optimization.
• Strategic system upgrades and sustainable management plans are crucial for enabling long-term computational and data science productivity.
• Continuous assessment and adaptation are key to addressing the unique challenges of optimizing HPC for scientific research.