SparkRA: Enabling Big Data Scalability for the GATK RNA-seq Pipeline with Apache Spark

Area of Science:

• Genomics
• Bioinformatics
• Computational Biology

Background:

• RNA sequencing (RNA-seq) data is rapidly increasing, driving demand for efficient analysis techniques.
• Current RNA-seq variant calling pipelines, like GATK's, face computational bottlenecks limiting scalability for large datasets.
• Efficient analysis is crucial for applications such as genotype-phenotype relationship identification and result validation.

Purpose of the Study:

• To develop a scalable pipeline for RNA-seq variant calling using Apache Spark.
• To address the computational limitations of existing GATK RNA-seq best practices pipelines.
• To accelerate the analysis of large-scale RNA-seq data.

Main Methods:

• Developed SparkRA, an Apache Spark-based pipeline for GATK RNA-seq variant calling.
• Implemented parallel processing across multiple cores and compute nodes.
• Evaluated performance on a single node and a distributed cluster.

Main Results:

• SparkRA reduced computation time by approximately 4× on a single node (from >5 h to 1.3 h for 32 GB data).
• On a 16-node cluster, SparkRA achieved a 7.7× speedup compared to single-node performance.
• SparkRA demonstrated a 1.2× speed advantage over other scalable solutions with equivalent accuracy.

Conclusions:

• SparkRA offers a highly efficient and scalable solution for RNA-seq variant calling.
• The pipeline significantly overcomes computational limitations, enabling faster analysis of large genomic datasets.
• SparkRA enhances the accessibility and practicality of RNA-seq data analysis in various biological research areas.