PQSDC: a parallel lossless compressor for quality scores data via sequences partition and run-length prediction...

Area of Science:

• Bioinformatics
• Computational Biology
• Data Compression

Background:

• Quality Scores Data (QSD) constitutes a significant portion (70%) of compressed FastQ files from short and long reads sequencing.
• Efficient compression of QSD is crucial for large-scale genomics data sharing and long-term archival, balancing compression ratio, time, and memory.
• Existing compression methods may not optimally address the specific characteristics of QSD.

Purpose of the Study:

• To introduce a novel parallel lossless compression algorithm, PQSDC, specifically designed for Quality Scores Data (QSD).
• To achieve a balance between compression ratio, time efficiency, and memory consumption for QSD compression.
• To leverage multicore CPU clusters for highly concurrent compression and decompression processes.

Main Methods:

• Development of PQSDC, a parallel lossless QSD-dedicated compression algorithm.
• Implementation of a parallel sequences-partition model to reduce peak memory and time costs.
• Integration of a parallel four-level run-length prediction mapping model to enhance compression ratio.
• Designed for high concurrency using multicore CPU clusters.

Main Results:

• PQSDC demonstrated significant improvements over state-of-the-art methods on 27 real-world datasets.
• For short reads, PQSDC achieved up to 7.06% better average compression ratio and substantial time/memory savings (up to 79.96% and 68.34% respectively).
• For long reads, PQSDC showed up to 13.42% improvement in weighted average compression ratio with notable time/memory savings (up to 72.53% and 19.44% respectively).
• PQSDC ranked second in compression robustness, indicating resilience to variations in QSD distributions.

Conclusions:

• PQSDC offers a promising solution for parallel QSD compression, effectively balancing storage costs, time consumption, and memory usage.
• The algorithm's parallel design and specialized models contribute to its superior performance in compressing QSD.
• PQSDC is available for download, facilitating its adoption in genomics data management and archival workflows.