Practical implications of using non-relational databases to store large genomic data files and novel phenotypes

Area of Science:

• Bioinformatics
• Genomics
• Database Management

Background:

• Genomic and phenotypic data storage presents challenges due to diverse formats and large volumes.
• Efficient data management is crucial for genetic evaluations and research.

Purpose of the Study:

• To provide practical guidelines for storing genomic and phenotypic data using a non-relational database.
• To assess MongoDB's suitability for managing diverse genetic and unstructured phenotypic information.

Main Methods:

• Utilized MongoDB to store various genomic (FASTQ, VCF, PLINK) and phenotypic data, including media files.
• Compared the non-relational approach with a relational database for importing and querying performance.
• Employed both real and simulated datasets for comprehensive evaluation.

Main Results:

• MongoDB demonstrated sublinear query running times with increasing sample/marker numbers.
• The non-relational database effectively managed diverse data formats and facilitated searches within genomic regions.
• Relational databases showed faster import times for large SNP datasets (50M SNPs in PLINK format).

Conclusions:

• Non-relational databases like MongoDB offer advantages for managing diverse genomic and phenotypic data, particularly for query efficiency.
• Data conversion is a significant, costly step in preparing genomic data for both relational and non-relational database systems.
• Careful consideration of data conversion is essential for the successful implementation of large-scale genomic data management solutions.