A benchmark study of compression software for human short-read sequence data

Area of Science:

• Genomics
• Bioinformatics
• Data Science

Background:

• Whole genome sequencing generates massive datasets, necessitating efficient data compression for storage and transfer.
• Cost-effective management of genomic data is critical for large-scale research initiatives.

Purpose of the Study:

• To benchmark the performance of specialized compression tools for paired-end fastq.gz files.
• To compare Genozip's BAM file compression against SAMtools.
• To evaluate compression ratios, speed, and file format compatibility.

Main Methods:

• Benchmarking of DRAGEN ORA, Genozip, repaq, and SPRING on paired-end fastq.gz files from the genome-in-a-bottle consortium.
• Comparative analysis of Genozip and SAMtools for BAM file compression.
• Assessment of compression ratios, compression/decompression times, and file format support.

Main Results:

• All tested tools provided lossless compression.
• DRAGEN ORA and Genozip achieved compression ratios of approximately 1:6 for fastq.gz files.
• Repaq and SPRING showed lower compression ratios (1:2 and 1:4, respectively) and longer processing times.
• Genozip offered ~16% better compression for BAM files than SAMtools, though SAMtools produces widely compatible CRAM files.
• Genozip supports multiple file formats, unlike ORA, repaq, and SPRING which are limited to fastq.gz.

Conclusions:

• Specialized software effectively compresses paired-end short-read sequence data.
• Commercial tools generally provide higher compression ratios than freely available options.
• Genozip offers a balance of high compression, broad format support, and accessible source code, despite requiring a license.