L2G:为基因组学任务重新利用语言模型
Wenduo Cheng1, Junhong Shen2, Mikhail Khodak3
1Ray and Stephanie Lane Computational Biology Department, School of Computer Science, Carnegie Mellon University, Pittsburgh, PA 15213, USA.
bioRxiv : the preprint server for biology
|December 23, 2024
概括
为基因组学重新利用大型语言模型 (LLM) 绕过了数据和计算挑战. L2G方法使LLMs适应基因组任务,在没有广泛的DNA预训练的情况下实现更高的性能.
科学领域:
- 基因组学就是基因组学.
- 生物信息学是一种生物信息学.
- 计算生物学 计算生物学
背景情况:
- 基础模型 (FMs) 正在改变基因组学,反映了自然语言处理 (NLP) 的成功.
- 从头开始开发基因组FM是计算上昂贵的,需要大量高质量的数据集.
- 在NLP中,大型语言模型 (LLM) 从工业规模的数据和基础设施中受益.
研究的目的:
- 为基因组学适应现有的LLM,克服数据和计算瓶.
- 引入L2G,一种用于为各种基因组应用重新利用LLM的方法.
- 评估LLM适应在基因组学中的有效性.
主要方法:
- 利用从NLP转换器到基因组数据的跨模式转移.
- 使用神经架构搜索 (NAS) 来适应LLM架构.
- 采用一种新的基因组任务三阶段培训程序.
主要成果:
- 在测试的基因组学基准测试任务中,L2G在超过一半的任务中取得了卓越的性能.
- 该模型的性能优于微调的基因组FM和特定任务模型.
- 在增强剂活性预测中,L2G成功地识别了显著的转录因子动机.
结论:
- 预先训练的语言模型显示出了显著的可通用性,用于诸如基因组学之类的域外任务.
- L2G为开发基因组模型提供了一种高效,资源密集度较低的方法.
- 这项工作为利用基因组研究中的LLM开辟了新的途径.
相关概念视频
02:02Genomics
35.8K
Genomics is the science of genomes: it is the study of all the genetic material of an organism. In humans, the genome consists of information carried in 23 pairs of chromosomes in the nucleus, as well as mitochondrial DNA. In genomics, both coding and non-coding DNA is sequenced and analyzed. Genomics allows a better understanding of all living things, their evolution, and their diversity. It has a myriad of uses: for example, to build phylogenetic trees, to improve productivity and...
35.8K
03:36Genome Annotation and Assembly
18.8K
The genome refers to all of the genetic material in an organism. It can range from a few million base pairs in microbial cells to several billion base pairs in many eukaryotic organisms. Genome assembly refers to the process of taking the DNA sequencing data and putting it all back together in a correct order to create a close representation of the original genome. This is followed by the identification of functional elements on the newly assembled genome, a process called genome annotation.
18.8K
02:54Evolutionary Relationships through Genome Comparisons
5.7K
Genome comparison is one of the excellent ways to interpret the evolutionary relationships between organisms. The basic principle of genome comparison is that if two species share a common feature, it is likely encoded by the DNA sequence conserved between both species. The advent of genome sequencing technologies in the late 20th century enabled scientists to understand the concept of conservation of domains between species and helped them to deduce evolutionary relationships across diverse...
5.7K