医学CPT:具有大规模PubMed搜索日志的对比预训练变压器,用于零射击的生物医学信息检索
Qiao Jin1, Won Kim1, Qingyu Chen1
1National Center for Biotechnology Information (NCBI), National Library of Medicine (NLM), National Institutes of Health (NIH).
ArXiv
|October 1, 2025
概括
MedCPT是一种新型的对比预训练型变压器,可以增强生物医学信息检索 (IR),而不需要查询文章注释. 这个模型在零射击语义IR任务中实现了最先进的性能.
科学领域:
- 生物医学信息学 生物医学信息学
- 自然语言处理自然语言处理.
- 信息检索 信息检索
背景情况:
- 生物医学知识获取和临床决策支持严重依赖于有效的信息检索 (IR).
- 目前使用语言模型的语义检索方法需要广泛的查询-文章注释,这在生物医学领域很少.
- 因此,大多数现有的生物医学IR系统都依赖于词汇匹配,这限制了它们的有效性.
研究的目的:
- 介绍MedCPT,一个开创性的对比预训练型变压器模型,旨在在生物医学中实现零射击的语义信息检索.
- 为了应对有限的注释数据的挑战,用于培训生物医学IR的高级语言模型.
主要方法:
- 从PubMed.收集了2.55亿用户点击日志的大规模数据集.
- 采用对比学习来训练一个集成的回收者和重新排名模型 (MedCPT).
- 在六个生物医学IR任务中评估了MedCPT.
主要成果:
- 在六个生物医学IR任务中,MedCPT建立了新的最先进的性能.
- 性能优于现有的基线,包括显著更大的模型,如GPT-3大小的cpt-text-XL.
- 在生成用于语义评估的生物医学文章和句子表示中表现出卓越的性能.
结论:
- 在生物医学中,MedCPT为零射击语义信息检索提供了一个高度有效的解决方案.
- 该模型利用大规模点击日志数据的能力克服了注释稀缺性.
- MedCPT很容易适用于各种现实世界生物医学IR应用,并提供了改进的语义表示.
相关概念视频
Improving Translational Accuracy
3.5K
3.5K
Improving Translational Accuracy
14.1K
Base complementarity between the three base pairs of mRNA codon and the tRNA anticodon is not a failsafe mechanism. Inaccuracies can range from a single mismatch to no correct base pairing at all. The free energy difference between the correct and nearly correct base pairs can be as small as 3 kcal/ mol. With complementarity being the only proofreading step, the estimated error frequency would be one wrong amino acid in every 100 amino acids incorporated. However, error frequencies observed in...
14.1K
Tissue Transplantation
921
Tissue transplantation is a significant medical procedure involving the transfer of cells, tissues, or organs from a donor to a recipient, with the primary aim of restoring lost functions. This procedure is crucial in treating a broad spectrum of diseases, including kidney diseases, liver failure, heart disease, and certain types of cancers.
The Biology of Tissue Transplantation
The biology of tissue transplantation hinges on the Major Histocompatibility Complex (MHC) molecules. These molecules...
The Biology of Tissue Transplantation
The biology of tissue transplantation hinges on the Major Histocompatibility Complex (MHC) molecules. These molecules...
921
Bioequivalence: Overview
1.8K
Pharmaceutical equivalents, by definition, are drug products with the same active ingredient in the same quantities, encapsulated in identical dosage forms, and intended for the same administration routes. These pharmaceutical equivalents are deemed bioequivalent if the bioavailability of the active entity in the drug preparations is similar. Moreover, pharmaceutical equivalents demonstrating bioequivalence are also regarded as therapeutically equivalent. This means that when used as directed,...
1.8K
Blood Transfusion
2.1K
Blood transfusion is a critical medical procedure that saves lives and treats various medical conditions. It involves transferring blood from a donor to a recipient. This process requires a thorough understanding of the ABO blood group system and its associated antigens and antibodies.
Blood Transfusion Overview
A blood transfusion is a medical procedure used to replace blood lost due to injury, surgery, or to treat conditions such as anemia or cancer. During a transfusion, donor blood is...
Blood Transfusion Overview
A blood transfusion is a medical procedure used to replace blood lost due to injury, surgery, or to treat conditions such as anemia or cancer. During a transfusion, donor blood is...
2.1K
Clinical Trials
10.1K
Clinical trials are prospective experimental studies conducted on humans to determine the safety and efficacy of treatments, drugs, diet methods, and medical devices. Using statistics in clinical trials enables researchers to derive reasonable and accurate conclusions from the collected data, allowing them to make wise decisions in uncertain situations. In medical research, statistical methods are crucial for preventing errors and bias.
There are four phases in a clinical trial. A phase one...
There are four phases in a clinical trial. A phase one...
10.1K


