大型语言模型知识的可靠性跨品牌和通用癌症药物名称
Jack Gallifant1,2, Shan Chen1,2,3, Sandeep K Jain4,5
1Artificial Intelligence in Medicine (AIM) Program, Mass General Brigham, Harvard Medical School, Boston, MA.
JCO clinical cancer informatics
|June 16, 2025
概括
大型语言模型 (LLM) 在匹配瘤药物名称方面表现出很高的准确性,但在复杂的临床任务中扎. 它们的性能在品牌和通用名称之间有很大差异,影响了患者的护理.
科学领域:
- 人工智能在医学中的应用
- 临床信息学 临床信息学
- 瘤学药物命名法 药物命名法
背景情况:
- 大型语言模型 (LLM) 在医疗保健中越来越多地使用,但其与微妙的医学术语 (如瘤学药物名称) 的性能尚未完全理解.
- 药物命名的变化 (品牌与仿制药) 可能会影响LLM的输出,可能会影响临床决策和患者安全.
研究的目的:
- 评估领先的LLM在各种临床应用中的品牌和通用瘤药物名称的性能和一致性.
- 在瘤学LLM输出中识别与药物命名相关的潜在偏见或性能差异.
主要方法:
- 评估了三个LLM (GPT-3.5-turbo-0125,GPT-4-turbo,GPT-4o) 使用来自HemOnc本体学的瘤学药物名称的综合数据集.
- 评估了LLM在药物名称识别,词汇关联,药物相互作用 (DDI) 检测和免疫相关不良事件 (irAE) 诊断方面的绩效,使用品牌和仿制药名称.
主要成果:
- 在品牌和通用名称匹配方面,LLMs取得了很高的准确性 (例如,GPT-4o:品牌名称的97.38%).
- 在词汇关联和情绪分析中观察到显著的不一致性,GPT-3.5-turbo-0125显示对品牌名称的偏见.
- 药物相互作用检测准确度在所有模型中都很低 (<26%),品牌和通用名称之间没有显著差异. 免疫相关不良事件的诊断一致性各不相同.
结论:
- 虽然在名称识别方面很熟练,但在复杂的临床任务中,LLM在处理品牌与通用瘤药物名称时表现出严重的不一致性.
- 这些发现强调了加强LLM稳定性评估和开发更一致的系统来处理临床环境中的药物命名规范变化的必要性.
相关概念视频
Drug Nomenclature
2.3K
During the development of a new pharmaceutical, the manufacturer initially assigns a code name to the drug. Once approved, the drug receives a United States Adopted Name (USAN)—a generic, nonproprietary designation. Upon being listed in the United States Pharmacopeia, this nonproprietary name becomes the drug's official name. Additionally, the manufacturer assigns a proprietary name or trademark, which serves as the brand name under which the drug is marketed. It is worth noting that...
2.3K
Treatment Resistant Cancers
3.4K
Cancer is the second leading cause of death in the United States. A cancer cell is genetically unstable and hence can mutate faster. They can also modify their microenvironment and escape immune surveillance. The difficulties in treating cancer are further compounded by the emergence of rapid resistance to anticancer drugs. The most common ways to attain resistance in cancer cells include alteration in drug transport and metabolism, modification of drug target, elevated DNA damage response, or...
3.4K
Combination Therapies and Personalized Medicine
5.1K
Combining two or more treatment methods increases the life span of cancer patients while reducing damage to vital organs or tissue from the overuse of a single treatment. Combination therapy also targets different cancer-inducing pathways, thus reducing the chances of developing resistance to treatment.
The combination of the drug acetazolamide and sulforaphane is a good example of combination therapy to treat cancer. The cells in the interior of a large tumor often die due to the hypoxic and...
The combination of the drug acetazolamide and sulforaphane is a good example of combination therapy to treat cancer. The cells in the interior of a large tumor often die due to the hypoxic and...
5.1K
Drug Discovery: Overview
8.8K
Drug discovery is a multifaceted process involving extensive screening, testing, and optimization of lead compounds to identify potential new drugs for therapeutic use. It combines several approaches, including screening large numbers of natural products, chemical modification of known active molecules, identification of new drug targets, and rational design based on biological mechanisms and drug-receptor structure. These approaches are carried out in both academic research laboratories and...
8.8K
Targeted Cancer Therapies
7.8K
The targeted cancer therapies, also known as “molecular targeted therapies,” take advantage of the molecular and genetic differences between the cancer cells and the normal cells. It needs a thorough understanding of the cancer cells to develop drugs that can target specific molecular aspects that drive the growth, progression, and spread of cancer cells without affecting the growth and survival of other normal cells in the body.
There are several types of targeted therapies against...
There are several types of targeted therapies against...
7.8K
Cancer-Critical Genes II: Tumor Suppressor Genes
8.2K
Genes usually encode proteins necessary for the proper functioning of a healthy cell. Mutations can often cause changes to the gene expression pattern, thereby altering the phenotype.
When the function of certain critical genes, especially those involved in cell cycle regulation and cell growth signaling cascades, gets disrupted, it upsets the cell cycle progression. Such cells with unchecked cell cycles start proliferating uncontrollably and eventually develop into tumors.
Such genes that act...
When the function of certain critical genes, especially those involved in cell cycle regulation and cell growth signaling cascades, gets disrupted, it upsets the cell cycle progression. Such cells with unchecked cell cycles start proliferating uncontrollably and eventually develop into tumors.
Such genes that act...
8.2K


