带有伪标签的半监督学习与监管序列预测的大型语言模型相比比较好
Han Phan1, Céline Brouard1, Raphaël Mourad1,2
1INRAE, MIAT, 31326 Castanet-Tolosan, France.
Briefings in bioinformatics
|November 3, 2024
概括
本研究引入了一种半监督式学习方法,用于从DNA序列中预测分子过程,利用未标记的基因组数据来改进非编码变体的深度学习模型.
科学领域:
- 基因组学就是基因组学.
- 生物信息学是一种生物信息学.
- 计算生物学 计算生物学
背景情况:
- 深度学习提供了来自全基因组关联研究 (GWAS) 的非编码单核酸多态 (SNP) 的生物学见解.
- 监督学习方法受到DNA序列功能数据稀缺性的限制.
- 哺乳动物DNA序列数据丰富,但往往缺乏相关的功能信息.
研究的目的:
- 开发一种新的半监督学习 (SSL) 方法,以克服监督学习在从DNA中预测分子过程方面的局限性.
- 通过有效利用大量未标记的DNA序列来增强模型预训练.
- 提高预测性能,特别是在功能数据有限的场景中,例如预测转录因子结合.
主要方法:
- 提出了一种基于伪标记的新型半监督学习 (SSL) 方法,用于对未标记的DNA序列进行预训练模型.
- 纳入了来自Noisy Student算法的原则,以在预训练期间完善伪标记数据的信心.
- 灵活的方法允许训练各种神经网络架构,包括先进的模型.
主要成果:
- 与标准的监督学习相比,SSL方法在各种任务中显示出预测性能的显著改善.
- 该方法在预测转录因子结合位点方面表现特别有效,即使训练数据非常有限.
- 使用SSL训练的小型模型实现了与DNABERT2.2等大型语言模型相比或超过的性能.
结论:
- 半监督学习,特别是伪标签和杂学生原则,是基因组学深度学习的强大策略.
- 这种方法有效地利用丰富的未标记的DNA序列来增强分子过程的预测建模.
- SSL为监督学习提供了灵活和高性能的替代方案,特别是当功能性基因组数据稀缺时.
相关概念视频
02:02Cis-regulatory Sequences
9.8K
Cis-regulatory sequences are short fragments of non-coding DNA that are present on the same chromosomes as the genes that they regulate. These fragments serve as binding sites for transcriptional regulators, proteins that are responsible for controlling gene transcription and differential gene expression across cell types in eukaryotes. Cis-regulatory sequences can be close to the gene of interest or thousands of bases away in the DNA sequence; however, those sequences that are further away are...
9.8K
02:13Cooperative Binding of Transcription Regulators
6.4K
Transcriptional regulators bind to specific cis-regulatory sequences in the DNA to regulate gene transcription. These cis-regulatory sequences are very short, usually less than ten nucleotide pairs in length. The short length means that there is a high probability of the exact same sequence randomly occurring throughout the genome. Since regulators can also bind to groups of similar sequences, this further increases the chances of random binding. Transcriptional regulators form...
6.4K
02:24Regulation of Expression Occurs at Multiple Steps
3.0K
3.0K
02:23Master Transcription Regulators
6.9K
Master transcription regulators are regulatory proteins that are predominantly responsible for regulating the expression of multiple genes. Often these genes work in concert to drive a complex process. Activation of a master transcription regulator can lead to a cascade of transcriptional activation necessary for that outcome. These regulators can directly bind to the regulatory sequences of the various genes involved, or they can indirectly regulate transcription by binding to regulatory...
6.9K
01:23Regulation of Expression at Multiple Steps
873
The gene expression in cells is regulated at different stages: (i) transcription, (ii) RNA processing, (iii) RNA localization, and (iv) translation. Transcriptional regulation is mediated by regulatory proteins such as transcription factors, activators, or repressors—these control gene expression by initiating or inhibiting the transcription of genes. Once a precursor or pre-mRNA is produced, it undergoes post-transcriptional modification, including 5' capping, splicing, and the...
873
02:39lncRNA - Long Non-coding RNAs
8.5K
In humans, more than 80% of the genome gets transcribed. However, only around 2% of the genome codes for proteins. The remaining part produces non-coding RNAs which includes ribosomal RNAs, transfer RNAs, telomerase RNAs, and regulatory RNAs, among other types. A large number of regulatory non-coding RNAs have been classified into two groups depending upon their length – small non-coding RNAs, such as microRNA, which are less than 200 nucleotides in length, and long non-coding RNA...
8.5K