Compression of Large genomic datasets using COMRAD on Parallel Computing Platform.

Area of Science:

• Bioinformatics
• Computational Biology
• Genomics

Background:

• The post-genomic era presents significant big data storage challenges.
• High-performance computing (HPC) solutions are crucial for managing large-scale genomic datasets.
• Efficient archiving of vast sequence data requires effective data compression strategies.

Purpose of the Study:

• To describe a parallel-computing approach for genomic data compression.
• To reduce the on-disk storage footprint of large genomic sequence volumes.
• To enhance the computational infrastructure for efficient data archiving.

Main Methods:

• A parallel-computing approach utilizing a message-passing library was developed.
• The compression stages were distributed across compute clusters.
• The method was applied to 21 Eukaryotic genomes using stratified sampling.

Main Results:

• The genomic compression approach significantly reduces on-disk storage.
• An average disk space reduction of 6-fold was achieved.
• Compression time was three times faster compared to the COMRAD method.

Conclusions:

• The described parallel-computing approach offers an efficient solution for genomic data compression.
• This method effectively addresses big data storage challenges in the post-genomic era.
• The approach supports more efficient archiving and computational infrastructure for genomic data management.