探索变压器模型:微调VS推断,从生物医学文本中提取关系
Hajar El Janah1, Youness Nachid-Idrissi1, Mourad Sarrouti2
1Laboratory of Intelligent Systems and Applications, Faculty of Sciences and Techniques, Sidi Mohamed Ben Abdellah University, Fez, Morocco.
Computational and structural biotechnology journal
|January 15, 2026
概括
精心调整的变压器模型在生物医学关系提取方面表现优于生成AI,实现了两倍的性能. 特定领域的预培训显著提高了生成模型的能力,突出显示了人工智能应用中需要专门数据的需求.
科学领域:
- 生物医学信息学是生物医学信息学.
- 人工智能的人工智能
- 自然语言处理自然语言处理.
背景情况:
- 生物医学数据的数量正在迅速增加,因此手动提取信息是不可行的.
- 生物医学关系提取自动化在文本中发现关系,对于知识发现至关重要.
- 现有的变压器模型需要昂贵的,专家创建的数据集进行微调.
研究的目的:
- 评估生成人工智能 (GenAI) 模型用于生物医学关系提取的可靠性.
- 将微调的变压器模型的性能与各种生成的大型语言模型 (LLM) 进行比较.
- 评估特定领域预培训对生成LLM绩效的影响.
主要方法:
- 与微调的变压器模型进行比较 (T5,PubMedBERT等). 具有生成的LLM (米斯特拉-7B,LLaMA2-7B,LLaMA3-8B,杰玛,RAG,Me-LLaMA-13B) 的.
- 评估了四个关键生物医学关系提取任务的模型:化学蛋白,疾病蛋白,药物相互作用和蛋白质相互作用.
- 使用相同的数据集进行微调和生成模型实验.
主要成果:
- 与生成型LLM (36.64-53.94) 相比,微调的变压器模型获得了明显更高的分数 (84.42-90.35).
- 生成型LLM的性能大约是微调模型的一半.
- 在MIMIC-III上进行预训练的Me-LLaMA比一般领域的预训练模型表现更好 (45.76),证明了专业预训练的价值.
结论:
- 微调的变压器模型仍然优于生物医学关系提取任务.
- 生成型的LLM显示出潜力,但需要大量的领域特定预培训,以获得竞争性表现.
- 特定领域的预培训对于提高生物医学等专业领域的生成模型的有效性至关重要.
相关概念视频
02:07Improving Translational Accuracy
3.5K
3.5K
02:07Improving Translational Accuracy
14.1K
Base complementarity between the three base pairs of mRNA codon and the tRNA anticodon is not a failsafe mechanism. Inaccuracies can range from a single mismatch to no correct base pairing at all. The free energy difference between the correct and nearly correct base pairs can be as small as 3 kcal/ mol. With complementarity being the only proofreading step, the estimated error frequency would be one wrong amino acid in every 100 amino acids incorporated. However, error frequencies observed in...
14.1K
01:25Transformers with Off-Nominal Turns Ratios
519
In scenarios involving parallel transformers with disparate ratings, developing per-unit models requires accommodating off-nominal turns ratios. This situation arises when the selected base voltages are not proportional to the transformer’s voltage ratings. Consider a transformer where the rated voltages are related by the term a. If the chosen voltage bases satisfy a relationship involving term b, term c is defined as the ratio of these bases. This ratio is then substituted into the...
519
01:31Transducer Mechanism: Nuclear Receptors
2.4K
Nuclear receptors, or NRs, are unique transcription factors that regulate gene transcription and affect the cellular pathways involved in reproduction, development, or metabolism. Their ability to be stimulated by small lipophilic ligands and control vital cellular processes makes them ideal drug targets. Nearly 10-15% of currently prescribed drugs target these receptors.
About 48 different soluble family members of nuclear receptors are identified that can be divided into two main classes:
About 48 different soluble family members of nuclear receptors are identified that can be divided into two main classes:
2.4K