redbiom: a Rapid Sample Discovery and Feature Characterization System.

Area of Science:

• Microbiome research
• Bioinformatics
• Computational biology

Background:

• Whole-community microbiome analyses reveal broad ecological patterns.
• Identifying specific microbial features driving these patterns has been challenging.
• Large-scale DNA sequencing generates vast datasets, necessitating efficient data retrieval.

Purpose of the Study:

• Introduce redbiom, a microbiome caching layer for rapid sample querying.
• Enable searching for samples based on specific features or metadata.
• Facilitate new types of microbiome characterization and discovery.

Main Methods:

• Implemented redbiom using an in-memory NoSQL database (Redis).
• Developed functionality to query samples by features, retrieve data/metadata, and filter by metadata values.
• Integrated redbiom with QIIME 2 and made it open-source.

Main Results:

• Redbiom allows rapid querying of over 100,000 public microbiome samples.
• The caching server requires minimal resident memory (35 GB for >100,000 samples).
• Enables identification of environments with specific unannotated microbes and correlates between features and metadata.

Conclusions:

• Redbiom dramatically accelerates the search for microbiome samples containing specific features.
• Facilitates rapid discovery of new microbiome relationships and correlations.
• Supports large-scale microbiome analyses and can be deployed for proprietary databases.