GenVarLoader: An accelerated dataloader for applying deep learning to personalized genomics.

Area of Science:

• Genomics
• Bioinformatics
• Machine Learning

Background:

• Deep learning sequence models show promise for variant effect prediction using personalized genomics data.
• Current applications are hindered by substantial computational demands for data storage and retrieval.

Purpose of the Study:

• To introduce GenVarLoader, a novel system designed to overcome computational limitations in personalized genomics data handling.
• To improve the efficiency of deep learning models for variant effect prediction.

Main Methods:

• Development of GenVarLoader utilizing novel memory-mapped formats for personalized genomic data.
• Optimization of data locality for efficient storage and retrieval.
• Benchmarking against existing data storage and access methods.

Main Results:

• GenVarLoader achieves approximately 1,000x faster throughput compared to existing alternatives.
• The system demonstrates approximately 2,000x better compression ratios.
• Enables more efficient application of deep learning models on large genomic datasets.

Conclusions:

• GenVarLoader significantly enhances the computational efficiency of processing personalized genomic data.
• The tool facilitates the practical application of advanced deep learning models for variant effect prediction.
• Addresses key bottlenecks in genomics data management for machine learning applications.