An efficient classification algorithm for NGS data based on text similarity.

Area of Science:

• Bioinformatics
• Computational Biology
• Genomics

Background:

• High-throughput sequencing generates massive datasets, challenging current data processing and storage capabilities.
• Data redundancy in sequencing data increases computational demands and can introduce noise.
• Efficient data clustering is vital for managing large sequencing datasets and improving downstream analysis.

Purpose of the Study:

• To introduce a high-performance short sequence classification algorithm (HSC) for next-generation sequencing (NGS) data.
• To address the computational challenges posed by the increasing volume of sequencing data.
• To improve the efficiency of data processing and reduce resource consumption in genomic analyses.

Main Methods:

• HSC algorithm utilizes k-mers and a hash table for efficient data organization.
• It identifies and merges duplicated and reverse complementary k-mers to create unique sets.
• Text similarity is employed to iteratively merge read clusters until convergence.
• The final clusters represent groups of similar short sequences.

Main Results:

• HSC successfully clustered 100 million short reads in under 2 hours.
• The algorithm demonstrated significant reductions in memory consumption compared to existing methods.
• HSC proved to be substantially faster than other tools for handling large-scale sequencing data.
• When used as a preprocessing step, HSC greatly reduced memory and time for assembly tools.

Conclusions:

• HSC offers a highly efficient solution for clustering large volumes of NGS data.
• The algorithm effectively reduces computational resource requirements for sequence data processing.
• HSC enhances the performance of downstream genomic assembly tasks, leading to improved assembly metrics.