Streaming histogram sketching for rapid microbiome analytics

Area of Science:

• Genomic Research
• Microbiome Analysis
• Bioinformatics

Background:

• Publicly available microbiome data has surged, necessitating efficient analytical tools.
• The increasing volume of microbiome data and research requires rapid processing methods.
• Clinical metagenomics demands analytics capable of handling massive datasets quickly.

Purpose of the Study:

• To propose a novel method for compact representation of microbiome sequencing data.
• To enable rapid dissimilarity estimation, catalogue searching, and classification of microbiome samples.
• To address the need for fast analytics in the era of clinical metagenomics.

Main Methods:

• Applied streaming histogram sketching for dimensionality reduction of microbiome k-mer spectra.
• Created compressed 'histosketches' for efficient microbiome data representation.
• Utilized locality-sensitive hashing indexing for rapid similarity searches.

Main Results:

• Hitosketches enabled clustering of samples by type using Jaccard similarity.
• Demonstrated accurate classification of microbiome samples using machine learning classifiers.
• Achieved 97% accuracy in predicting antibiotic treatment in neonates from microbiome data.

Conclusions:

• The proposed method offers a new approach for rapid processing of microbiome data streams.
• Hitosketches allow for fast sample clustering, indexing, and classification.
• The HULK implementation efficiently processes large microbiome datasets on standard hardware.