mim:一个轻量级的辅助索引,可以实现快速,并行,gzipped FASTQ解析
bioRxiv : the preprint server for biology
|December 15, 2025
概括
一个名为mim的新辅助索引加快了压缩FASTQ文件的解析速度. 这种索引创建是一次性操作,可以快速并行处理基因组学数据,而不会改变原始文件.
科学领域:
- 生物信息学是一种生物信息学.
- 计算生物学 计算生物学
- 基因组学就是基因组学.
背景情况:
- FASTQ文件格式是生物信息学数据的标准.
- Gzip压缩的FASTQ文件 (fastq.gz) 在现代基因组学中存在可扩展性瓶.
- 高效分析大型压缩序列数据至关重要.
研究的目的:
- 介绍mim,一个轻量级的辅助索引,用于快速并行解析压缩的FASTQ文件.
- 解决处理大型基因组学数据集的可扩展性挑战.
- 提供一种方法来加速数据摄入和解析.
主要方法:
- 开发了mim,这是fastq.gz文件的辅助索引.
- 在各种FASTQ数据集上构建基准mim指数.
- 测量线程缩放用于mim指数辅助的并行FASTQ解析.
主要成果:
- 索引创建的开销最小 (∼20%) 并且需要很少的内存.
- 模拟索引很小 (通常是压缩文件大小的1/10).
- mim 能够大规模加速对gzipped FASTQ数据进行解析,并具有近线性线程缩放.
结论:
- mim为解析压缩的FASTQ文件提供了一个可扩展的解决方案.
- 该指数是附加的,不需要修改现有数据.
- mim显著提高了基因组学数据处理管道的效率.
相关概念视频
Sanger Sequencing
772.8K
DNA sequencing is a fundamental technique that is routinely used in the biological sciences. This method can be applied to a range of questions at different scales - from the sequencing of a cloned DNA fragment or the study of a mutation in a gene up to whole-genome sequencing. However, despite the widespread use of sequencing today, it was not until 1977 that Fredrick Sanger and his collaborators developed the chain-termination method to decode DNA sequences. It relies on the separation of a...
772.8K
Maxam-Gilbert Sequencing
12.5K
In the same year as the discovery of the Sanger sequencing method, another group of scientists, Allan Maxam and Walter Gilbert, demonstrated their chemical-cleavage method for DNA sequencing. The Maxam-Gilbert method relies on using different chemicals that can cleave the DNA sequence at specific sites, the separation of resulting DNA fragments of variable size using electrophoresis, and deciphering the DNA sequence from the resulting gel bands.
Challenges of the Maxam-Gilbert Method
The...
Challenges of the Maxam-Gilbert Method
The...
12.5K
Multi-species Conserved Sequences
4.6K
Next-generation sequencing technologies have created large genomic databases of a variety of animals and plants. Ever since the human genome project was completed, scientists studied the genome of primates, mammals, and other phylogenetically distant living beings. Such large-scale studies have provided new insights into the evolutionary relationship between organisms.
Although the genome of each species varies greatly from each other, a few sequences are highly conserved. Such conserved...
Although the genome of each species varies greatly from each other, a few sequences are highly conserved. Such conserved...
4.6K
Next-generation Sequencing
97.6K
The first human genome sequencing project cost $2.7 billion and was declared complete in 2003, after 15 years of international cooperation and collaboration between several research teams and funding agencies. Today, with the advent of next-generation sequencing technologies, the cost and time of sequencing a human genome have dropped over 100 fold.
Next-Generation Sequencing Methods
Although all next-generation methods use different technologies, they all share a set of standard features....
Next-Generation Sequencing Methods
Although all next-generation methods use different technologies, they all share a set of standard features....
97.6K
Mass Analyzers: Overview
1.5K
The mass analyzer is a crucial component of the mass spectrometer. In the ionization chamber, the vaporized sample is bombarded with a high-energy electron beam to generate a radical cation and further fragment into neutral molecules, radicals, and cations. A series of negatively charged accelerator plates accelerate the cations into the mass analyzer. The mass analyzer separates ions according to their mass-to-charge (m/z) ratios and then directs them to the detector. The common types of mass...
1.5K
Parallel Processing
600
The brain processes sensory information rapidly due to parallel processing, which involves sending data across multiple neural pathways at the same time. This method allows the brain to manage various sensory qualities, such as shapes, colors, movements, and locations, all concurrently. For instance, when observing a forest landscape, the brain simultaneously processes the movement of leaves, the shapes of trees, the depth between them, and the various shades of green. This enables a quick and...
600


