RabbitSketch: a high-performance sketching library for genome analysis.

Area of Science:

• Bioinformatics
• Computational Biology
• Data Science

Background:

• Efficient analysis of large-scale genomic data is crucial for biological research.
• Existing sketching algorithms can be computationally intensive, limiting scalability.
• Optimizing algorithms for modern multi-core processors is essential for faster data processing.

Purpose of the Study:

• To introduce RabbitSketch, a highly optimized library for sketching algorithms.
• To leverage multi-core CPU capabilities for enhanced performance in data analysis.
• To provide flexible Python and C++ interfaces for broad accessibility.

Main Methods:

• Implementation of optimized sketching algorithms including MinHash, OrderMinHash, and HyperLogLog.
• Exploitation of modern multi-core CPU architectures for parallel processing.
• Integration of RabbitSketch's Kssd algorithm into RabbitTClust for performance enhancement.

Main Results:

• RabbitSketch achieves significant speedups, ranging from 2.30× to 49.55×, over existing implementations.
• Analysis of 455GB genomic data completed in 5 minutes using RabbitSketch with minimal Python code.
• Enhanced RabbitTClust demonstrated a 1.54× speedup without compromising accuracy.

Conclusions:

• RabbitSketch offers a powerful and efficient solution for large-scale genomic data similarity analysis.
• The library's performance gains and ease of use facilitate faster and more scalable research.
• Optimized sketching algorithms are vital for advancing computational biology and bioinformatics.