Hummingbird: efficient performance prediction for executing genomic applications in the cloud

Area of Science:

• Bioinformatics
• Computational Biology
• Genomic Data Analysis

Background:

• Genomic applications on cloud platforms lack tools for predicting optimal instance types.
• Resource over- or under-matching leads to wasted time and money.
• Accurate instance selection is crucial for efficient genomic data processing.

Purpose of the Study:

• Introduce Hummingbird, a novel tool for predicting the performance of cloud computing instances.
• Enable users to select the most appropriate instance types for genomic applications based on memory and CPU.
• Facilitate cost and time savings in cloud-based genomic data analysis.

Main Methods:

• Hummingbird predicts performance for computing instances with varying memory and CPU.
• The tool supports applications specified in JSON or Workflow Description Language (WDL) formats.
• Experiments were conducted on major genomic data pipelines: GATK HaplotypeCaller, GATK Mutect2, and ENCODE ATAC-seq.

Main Results:

• Hummingbird accurately predicted the fastest, cheapest, and most cost-efficient compute instances.
• The tool demonstrated effectiveness across three major genomic data pipelines.
• Predictions were made in an economic manner, optimizing resource utilization.

Conclusions:

• Hummingbird provides an effective solution for selecting optimal cloud instances for genomic applications.
• The tool addresses the critical need for resource matching in cloud computing environments.
• Hummingbird contributes to more efficient and cost-effective execution of genomic workflows.