Distributed hybrid-indexing of compressed pan-genomes for scalable and fast sequence alignment

Area of Science:

• Genomics
• Bioinformatics
• Computational Biology

Background:

• Computational pan-genomics enables discovery of genetic variation across diverse populations.
• Rapid growth in whole-genome sequencing (WGS) data necessitates efficient compression and indexing.
• Existing compression methods are often slow and do not support efficient pan-genome analysis.

Purpose of the Study:

• To develop a scalable, distributed compressed hybrid-indexing method for large genomic datasets.
• To enable efficient pan-genome-based sequence search and read alignment.
• To improve the speed and efficiency of genomic data analytics.

Main Methods:

• Proposed a scalable distributed compressed hybrid-indexing method named DHPGIndex.
• Utilized Apache Spark for distributed computing experiments on a 448-core cluster.
• Evaluated performance using human and bacterial genome datasets of varying sizes.

Main Results:

• DHPGIndex achieved high compression ratios (e.g., 870:1 for BLAST index on 250 human genomes).
• Demonstrated efficient sequence alignment, with Bowtie2 aligning 14.6 GB of reads in 31.7 minutes.
• Successfully compressed and indexed large databases like GenBank (488 GB) with significant compression.

Conclusions:

• DHPGIndex offers a scalable and efficient solution for indexing and analyzing large-scale pan-genomic data.
• The method significantly reduces index size and accelerates sequence alignment.
• Enables faster and more effective comparative genomics and population genetics studies.