为促进者预测优化基因组语言模型:对代币化和跨物种学习进行比较研究
Eyal Hadad1, Noia Kogman1, Lina Golan1
1Faculty of Computer and Information Science, Ben-Gurion University of the Negev, David Ben-Gurion Blvd. 1, Beer-Sheva 8410501, Israel.
NAR genomics and bioinformatics
|March 16, 2026
概括
用于基因组学的大型语言模型 (LLM) 最好使用非重叠的k-mers进行促销者分类. 进化转移学习可以提高表现,特别是在有限的基因组数据下.
科学领域:
- 基因组学就是基因组学.
- 生物信息学是一种生物信息学.
- 机器学习 机器学习
背景情况:
- 大型语言模型 (LLM) 在基因组分析中表现有前途.
- 挑战包括代币化,评估和LLM应用中的数据稀缺.
研究的目的:
- 系统地评估使用LLMs的促销商分类的代币化方法.
- 评估负数据策略和转移学习对模型性能的影响.
主要方法:
- 评估了四种代币化方法:非重叠的6-mer,重叠的6-mer,字节对编码 (BPE) 和WordPiece (WPC).
- 使用积极促销者混杂和随机非促销者片段负数据策略评估绩效.
- 采用位置式SHAP框架进行可解释性和进化信息转移学习.
主要成果:
- 不重叠的6mer标记化在八个生物体中表现优于BPE和WPC.
- 该模型学习了生物相关的定位模式,而不是从负数据中学习的文物.
- 对相关物种的转移学习显著提高了性能,特别是在低数据场景中.
结论:
- 代币化选择和负数据设计极大地影响了基因组学LLM的表现.
- 不重叠的k-mers对于促进者分类是有效的.
- 进化转移学习为数据稀缺的基因组任务提供了可行的策略.
更多相关视频
11:36Author Spotlight: An Integrated Workflow to Study the Promoter-Centric Spatio-Temporal Genome Architecture in Scarce Cell Populations
Published on: April 21, 2023
3.1K
08:54In vivo Application of the REMOTE-control System for the Manipulation of Endogenous Gene Expression
Published on: March 29, 2019
7.7K
相关概念视频
Improving Translational Accuracy
3.8K
3.8K
Improving Translational Accuracy
15.4K
Base complementarity between the three base pairs of mRNA codon and the tRNA anticodon is not a failsafe mechanism. Inaccuracies can range from a single mismatch to no correct base pairing at all. The free energy difference between the correct and nearly correct base pairs can be as small as 3 kcal/ mol. With complementarity being the only proofreading step, the estimated error frequency would be one wrong amino acid in every 100 amino acids incorporated. However, error frequencies observed in...
15.4K
The Eukaryotic Promoter Region
19.4K
The eukaryotic promoter region is a segment of DNA located upstream of a gene. It contains an RNA polymerase binding site, a transcription start site, and several cis-regulatory sequences. The proximal promoter region is located in the vicinity of the gene and has cis-regulatory sequences and the core promoter. The core promoter is the binding site for RNA polymerase and is usually located between -35 and +35 nucleotides from the transcription start site. The distal promoter regions are...
19.4K
The Eukaryotic Promoter Region
4.2K
4.2K
Genetic Lingo
117.0K
Overview
117.0K
Regulation of Expression at Multiple Steps
1.5K
The gene expression in cells is regulated at different stages: (i) transcription, (ii) RNA processing, (iii) RNA localization, and (iv) translation. Transcriptional regulation is mediated by regulatory proteins such as transcription factors, activators, or repressors—these control gene expression by initiating or inhibiting the transcription of genes. Once a precursor or pre-mRNA is produced, it undergoes post-transcriptional modification, including 5' capping, splicing, and the...
1.5K
