GenMPI: Cluster Scalable Variant Calling for Short/Long Reads Sequencing Data.

Area of Science:

• Genomics
• Bioinformatics
• High-Performance Computing

Background:

• High-throughput sequencing generates vast amounts of data, posing computational challenges for real-time analysis.
• Single-node processing is a bottleneck for timely clinical diagnosis from large sequencing datasets.

Purpose of the Study:

• To develop a scalable and efficient variant calling workflow for rapid genomic data processing.
• To leverage Message Passing Interface (MPI) for parallel computation on distributed systems.

Main Methods:

• Implemented a portable variant calling workflow (GenMPI) using MPI for parallel data processing.
• Optimized data streaming and utilized MPI Remote Memory Access (RMA) for distributed operations.
• Integrated various aligners (BWA-MEM, Minimap2) and variant callers (GATK HaplotypeCaller, DeepVariant, Octopus, Clair3) for short and long reads.

Main Results:

• GenMPI demonstrates near-linear scalability for alignment up to 64 nodes (8192 CPU cores).
• The workflow achieves a two-fold performance improvement over existing big data solutions and is 20% faster than other MPI implementations.
• Variant calling accuracy is maintained compared to single-node methods, with scalable performance up to 8 nodes.

Conclusions:

• GenMPI provides a flexible, portable, and high-performance solution for large-scale genomic variant calling.
• The workflow accelerates real-time clinical diagnosis by overcoming computational bottlenecks in processing sequencing data.
• The developed system enhances the efficiency of genomic data analysis across diverse sequencing platforms.