选择性状态空间模型在预测RNA-Seq读取覆盖率方面优于变压器
Ian Holmes1,2, Johannes Linder2, David Kelley2
1Department of Bioengineering, University of California, University Drive, Berkeley 94703.
bioRxiv : the preprint server for biology
|March 3, 2025
概括
与传统的变压器模型相比,像Mamba这样的状态空间模型在基因表达预测准确度方面提供了轻微但持续的改进. 虽然这些收益尚未促进下游SNP分类,但基于Mamba的模型对功能基因组学有很大的前景.
科学领域:
- 计算生物学和基因组学
- 在生物信息学中的机器学习
背景情况:
- 变压器模型是DNA基因表达预测的基础,但它们的训练耗时且昂贵.
- 以信号处理为灵感的替代模型,包括状态空间模型 (Mamba),里埃变换 (Hyena) 和波形变换 (MultiResNet),已经出现,以解决这些局限性.
研究的目的:
- 评估替代机器学习架构作为替代或补充注意力机制来预测基因表达.
- 在功能基因组学任务中比较卷积,注意力,海纳,曼巴和条纹架构模型的性能.
主要方法:
- 在Python和Jax/Flax中开发了"bilby"软件库,提供各种卷积,注意力和状态空间模型.
- 在基因表达数据上进行监督的多任务学习实验.
- 在GTEx eQTL数据集上使用预测准确度指标 (Pearson R,r^2) 和下游SNP分类来比较模型性能.
主要成果:
- 结合双向Mamba层的卷积模型比卷积注意力模型显示出小但一致的预测准确度改进 (3-4%皮尔森R,1-2%r^2).
- 观察到的最高收益是条纹建筑,它结合了Mamba和注意力层.
- 模型没有竞争力,MultiResNet太慢;Mamba的准确性收益没有显著改善SNP分类性能.
结论:
- 选择性状态空间模型,特别是Mamba和条纹Mamba架构,证明了在功能基因组学中改善基因表达预测的潜力.
- 对于这些任务,对基于Mamba的模型进行进一步的研究是有必要的,尽管SNP下游分类目前的局限性.
- "比尔比"图书馆和训练有素的模型可供公众使用,用于可复制的研究.
相关概念视频
03:21RNA-seq
9.9K
RNA sequencing, or RNA-Seq, is a high-throughput sequencing technology used to study the transcriptome of a cell. Transcriptomics helps to interpret the functional elements of a genome and identify the molecular constituents of an organism. Additionally, it also helps in understanding the development of an organism and the occurrence of diseases.
Before the discovery of RNA-seq, microarray-based methods and Sanger sequencing were used for transcriptome analysis. However, while...
Before the discovery of RNA-seq, microarray-based methods and Sanger sequencing were used for transcriptome analysis. However, while...
9.9K
02:24Ribosome Profiling
3.5K
Ribosome profiling or ribo-sequencing is a deep sequencing technique that produces a snapshot of active translation in a cell. It selectively sequences the mRNAs protected by ribosomes to get an insight into a cell’s translation landscape at any given point in time.
Applications of ribosome profiling
Ribosome profiling has many applications, including in vivo monitoring of translation inside a particular organ or tissue type and quantifying new protein synthesis levels.
The technique...
Applications of ribosome profiling
Ribosome profiling has many applications, including in vivo monitoring of translation inside a particular organ or tissue type and quantifying new protein synthesis levels.
The technique...
3.5K