BigSeqKit:一个平行的大数据工具包,可以在规模上处理FASTA和FASTQ文件
César Piñeiro1, Juan C Pichel1
1CiTIUS, Universidade de Santiago de Compostela, Santiago de Compostela 15782, Spain.
GigaScience
|July 31, 2023
概括
通过利用高性能计算,BigSeqKit可以加速处理大型序列数据 (FASTA/FASTQ文件). 与生物信息学分析的现有方法相比,这个工具包提供了显著的加快速度.
科学领域:
- 生物信息学是一种生物信息学.
- 计算生物学 计算生物学
- 基因组学就是基因组学.
背景情况:
- 高通量测序产生了大量的数据集,通常是FASTA/FASTQ格式.
- 由于序列处理的局限性,现有的工具难以高效地处理非常大的序列文件.
- 像seqkit这样的工具中的当前并行化仅限于单个节点上的几个线程.
研究的目的:
- 开发一个高效的工具包来处理大型FASTA/FASTQ文件.
- 为了克服现有的序列数据处理工具的可扩展性限制.
- 为了使序列数据更快地转化为生物见解.
主要方法:
- 使用了一种高性能计算-大数据框架.
- 从seqkit工具中并行和优化命令.
- 开发了BigSeqKit作为一个用户友好的工具包和生物信息图书馆.
主要成果:
- BigSeqKit展示了实质性的速度改进,通常比最先进的工具快几十到几百倍.
- 该工具包旨在有效并行处理千兆字节规模的测序文件.
- 在各种硬件平台上安装和使用BigSeqKit很容易.
结论:
- BigSeqKit是一个全面和超快的工具包,用于大规模的FASTA/FASTQ文件处理.
- 它显著提高了处理大规模测序数据的效率.
- 该工具包是公开可用的,用于生物信息学研究.
相关概念视频
RNA-seq
10.1K
RNA sequencing, or RNA-Seq, is a high-throughput sequencing technology used to study the transcriptome of a cell. Transcriptomics helps to interpret the functional elements of a genome and identify the molecular constituents of an organism. Additionally, it also helps in understanding the development of an organism and the occurrence of diseases.
Before the discovery of RNA-seq, microarray-based methods and Sanger sequencing were used for transcriptome analysis. However, while...
Before the discovery of RNA-seq, microarray-based methods and Sanger sequencing were used for transcriptome analysis. However, while...
10.1K
Next-generation Sequencing
91.5K
The first human genome sequencing project cost $2.7 billion and was declared complete in 2003, after 15 years of international cooperation and collaboration between several research teams and funding agencies. Today, with the advent of next-generation sequencing technologies, the cost and time of sequencing a human genome have dropped over 100 fold.
Next-Generation Sequencing Methods
Although all next-generation methods use different technologies, they all share a set of standard features....
Next-Generation Sequencing Methods
Although all next-generation methods use different technologies, they all share a set of standard features....
91.5K
Sanger Sequencing
754.8K
DNA sequencing is a fundamental technique that is routinely used in the biological sciences. This method can be applied to a range of questions at different scales - from the sequencing of a cloned DNA fragment or the study of a mutation in a gene up to whole-genome sequencing. However, despite the widespread use of sequencing today, it was not until 1977 that Fredrick Sanger and his collaborators developed the chain-termination method to decode DNA sequences. It relies on the separation of a...
754.8K
Maxam-Gilbert Sequencing
11.2K
In the same year as the discovery of the Sanger sequencing method, another group of scientists, Allan Maxam and Walter Gilbert, demonstrated their chemical-cleavage method for DNA sequencing. The Maxam-Gilbert method relies on using different chemicals that can cleave the DNA sequence at specific sites, the separation of resulting DNA fragments of variable size using electrophoresis, and deciphering the DNA sequence from the resulting gel bands.
Challenges of the Maxam-Gilbert Method
The...
Challenges of the Maxam-Gilbert Method
The...
11.2K


