使用Spark进行可扩展和可维护的分布式序列对齐
IEEE transactions on computational biology and bioinformatics
|August 14, 2025
概括
生物信息学面临着大型基因组数据集的挑战. SparkLeBLAST提供了一个可扩展,可维护的并行BLAST解决方案,提高了基因组分析性能和研究人员的可访问性.
科学领域:
- 生物信息学是一种生物信息学.
- 计算生物学 计算生物学
- 基因组学就是基因组学.
背景情况:
- 基因组数据正在指数级增长,挑战了像NCBI BLAST这样的传统生物信息工具.
- 现有的并行BLAST工具 (mpiBLAST,SparkBLAST) 在大数据集的可扩展性,可维护性或性能方面存在局限性.
研究的目的:
- 开发一个并行BLAST工具,将mpiBLAST的性能和可扩展性与SparkBLAST的简单性和可维护性相结合.
- 为科学家没有广泛的分布式计算经验,提供一个民主化可扩展基因组分析的工具.
主要方法:
- 推出了SparkLeBLAST,这是一个平行BLAST工具,利用Spark框架和高效的数据分区.
- 实施了一种新的数据分区方法,可以克服SparkBLAST在大型数据库方面的局限性.
主要成果:
- SparkLeBLAST表现出显著的性能改进,运行速度高达SparkBLAST的6.68倍.
- 在BLAST搜索组件中实现了对COVID-19基因组多样性分析的88.6×加快速度,使用128个计算节点将整体分类学赋值加快了20.9×.
结论:
- SparkLeBLAST为并行BLAST搜索提供了一个高性能,可扩展和可维护的解决方案.
- 该工具提高了对更广泛的科学研究人员进行大规模基因组分析的可访问性.
相关概念视频
Multi-species Conserved Sequences
4.3K
Next-generation sequencing technologies have created large genomic databases of a variety of animals and plants. Ever since the human genome project was completed, scientists studied the genome of primates, mammals, and other phylogenetically distant living beings. Such large-scale studies have provided new insights into the evolutionary relationship between organisms.
Although the genome of each species varies greatly from each other, a few sequences are highly conserved. Such conserved...
Although the genome of each species varies greatly from each other, a few sequences are highly conserved. Such conserved...
4.3K
RNA-seq
10.4K
RNA sequencing, or RNA-Seq, is a high-throughput sequencing technology used to study the transcriptome of a cell. Transcriptomics helps to interpret the functional elements of a genome and identify the molecular constituents of an organism. Additionally, it also helps in understanding the development of an organism and the occurrence of diseases.
Before the discovery of RNA-seq, microarray-based methods and Sanger sequencing were used for transcriptome analysis. However, while...
Before the discovery of RNA-seq, microarray-based methods and Sanger sequencing were used for transcriptome analysis. However, while...
10.4K
Per-Unit Sequence Models
116
An ideal Y-Y transformer, grounded through neutral impedances, displays per-unit sequence networks akin to those of a single-phase ideal transformer when subjected to balanced positive- or negative-sequence currents. These currents do not produce neutral currents, and their associated voltage drops.
Zero-sequence currents, which are identical in magnitude and phase, generate a neutral current, resulting in voltage drops across the neutral impedance and the low-voltage winding. If the...
Zero-sequence currents, which are identical in magnitude and phase, generate a neutral current, resulting in voltage drops across the neutral impedance and the low-voltage winding. If the...
116
Maxam-Gilbert Sequencing
11.5K
In the same year as the discovery of the Sanger sequencing method, another group of scientists, Allan Maxam and Walter Gilbert, demonstrated their chemical-cleavage method for DNA sequencing. The Maxam-Gilbert method relies on using different chemicals that can cleave the DNA sequence at specific sites, the separation of resulting DNA fragments of variable size using electrophoresis, and deciphering the DNA sequence from the resulting gel bands.
Challenges of the Maxam-Gilbert Method
The...
Challenges of the Maxam-Gilbert Method
The...
11.5K
Next-generation Sequencing
92.6K
The first human genome sequencing project cost $2.7 billion and was declared complete in 2003, after 15 years of international cooperation and collaboration between several research teams and funding agencies. Today, with the advent of next-generation sequencing technologies, the cost and time of sequencing a human genome have dropped over 100 fold.
Next-Generation Sequencing Methods
Although all next-generation methods use different technologies, they all share a set of standard features....
Next-Generation Sequencing Methods
Although all next-generation methods use different technologies, they all share a set of standard features....
92.6K
Sanger Sequencing
757.1K
DNA sequencing is a fundamental technique that is routinely used in the biological sciences. This method can be applied to a range of questions at different scales - from the sequencing of a cloned DNA fragment or the study of a mutation in a gene up to whole-genome sequencing. However, despite the widespread use of sequencing today, it was not until 1977 that Fredrick Sanger and his collaborators developed the chain-termination method to decode DNA sequences. It relies on the separation of a...
757.1K


