评估大型语言模型在药监文献选中的表现:一项比较研究
Dan Li1, Leihong Wu1, Mingfeng Zhang2
1Division of Bioinformatics and Biostatistics, National Center for Toxicological Research, U.S. Food and Drug Administration, Jefferson, AR, United States.
Frontiers in drug safety and regulation
|September 22, 2025
概括
大型语言模型 (LLM) 在自动化药监文献选,提高药物安全监测的可重复性和灵敏性方面表现有前途. 通过示例和推理来增强性能,尽管具体性需要进一步优化.
科学领域:
- 药物监督和人工智能 药物监督和人工智能
- 药物安全中的自然语言处理.
- 文学审查的计算方法
背景情况:
- 药物监测对于药物安全至关重要,包括监测不良事件和安全问题.
- 用于药监督的手册文献选是耗时的,并且在出版量增加方面遇到了困难.
- 需要自动化解决方案,以高效地处理大量的科学文献,以监督药物安全.
研究的目的:
- 评估大型语言模型 (LLM) 在自动化药监监文献选方面的有效性.
- 评估特定的LLM (GPT-3.5,GPT-4,Claude2) 在识别安全信号审查相关文章的性能.
- 确定影响药监文献分类中LLM准确性的因素.
主要方法:
- 利用N-shot学习和连锁思维推理与LLM来分类医学出版物.
- 使用可重现性,灵敏性和特异性等指标评估LLM绩效.
- 分析了提示工程,示例选择和推理解释对预测准确性的影响.
主要成果:
- 在文献选中,LLM表现出高可重复性 (93%) 和灵敏性 (97%),具有中等的特异性 (67%).
- 提供LLM的例子,包括摘要,标签和推理解释,提高了业绩.
- 影响结果的关键因素包括关键词选择,提示设计,示例平衡和推理清晰度.
结论:
- 药监法提供了一个有希望的方法来提高药监文献选的效率和有效性.
- 优化LLM配置,包括提示和培训数据,可以进一步提高药物安全监测的准确性.
- 这项研究支持制药监督自动化系统的开发,为更安全的药品做出贡献.
更多相关视频
05:47Evidence-based Knowledge Synthesis and Hypothesis Validation: Navigating Biomedical Knowledge Bases via Explainable AI and Agentic Systems
Published on: June 13, 2025
1.3K
12:18A Machine Learning Approach to Design an Efficient Selective Screening of Mild Cognitive Impairment
Published on: January 11, 2020
7.9K
相关概念视频
01:26Pharmacokinetic Models: Comparison and Selection Criterion
338
Physiological and compartmental models are valuable tools used in studying biological systems. These models rely on differential equations to maintain mass balance within the system, ensuring an accurate representation of the dynamic processes at play.
Physiological models take a detailed approach by considering specific molecular processes. They can predict drug distribution, metabolism, and elimination changes, providing a comprehensive understanding of how drugs interact with the body.
Physiological models take a detailed approach by considering specific molecular processes. They can predict drug distribution, metabolism, and elimination changes, providing a comprehensive understanding of how drugs interact with the body.
338
01:19Pharmacovigilance
1.6K
Post-marketing surveillance is a critical component of pharmaceutical regulation, often uncovering unanticipated adverse drug reactions (ADRs) once a drug is widely used over an extended period.
This process, termed pharmacovigilance, aims to detect, evaluate, and minimize harmful effects related to medication use. The data collection for pharmacovigilance depends on spontaneous reporting systems, where healthcare professionals or patients voluntarily report suspected ADRs.
In some cases, there...
This process, termed pharmacovigilance, aims to detect, evaluate, and minimize harmful effects related to medication use. The data collection for pharmacovigilance depends on spontaneous reporting systems, where healthcare professionals or patients voluntarily report suspected ADRs.
In some cases, there...
1.6K
01:12Analysis of Population Pharmacokinetic Data
682
Analysis of population pharmacokinetic data involves studying the behavior of drugs within diverse populations to understand their pharmacokinetic parameters. Traditional pharmacokinetic methods typically involve collecting samples from a few individuals and estimating these parameters. While these methods are commonly used, they have limitations in capturing the variability in drug response among individuals or heterogeneous populations. Population pharmacokinetics is employed to address these...
682
01:14Analysis Methods of Pharmacokinetic Data: Model and Model-Independent Approaches
494
Drug disposition in the body is a complex process and can be studied using two major approaches: the model and the model-independent approaches.
The model approach uses mathematical models to describe changes in drug concentration over time. Pharmacokinetic models help characterize drug behavior in patients, predict drug concentration in the body fluids, calculate optimum dosage regimens, and evaluate the risk of toxicity. However, ensuring that the model fits the experimental data accurately...
The model approach uses mathematical models to describe changes in drug concentration over time. Pharmacokinetic models help characterize drug behavior in patients, predict drug concentration in the body fluids, calculate optimum dosage regimens, and evaluate the risk of toxicity. However, ensuring that the model fits the experimental data accurately...
494
01:06Model Approaches for Pharmacokinetic Data: Distributed Parameter Models
242
Pharmacokinetic models are mathematical constructs that represent and predict the time course of drug concentrations in the body, providing meaningful pharmacokinetic parameters. These models are categorized into compartment, physiological, and distributed parameter models.
The distributed parameter models are specifically designed to account for variations and differences in some drug classes. This model is particularly useful for assessing regional concentrations of anticancer or...
The distributed parameter models are specifically designed to account for variations and differences in some drug classes. This model is particularly useful for assessing regional concentrations of anticancer or...
242
01:20Pharmacokinetic Models: Overview
1.9K
Pharmacokinetic models utilize mathematical analysis to achieve a detailed quantitative understanding of a drug's life cycle within the body. They are instrumental in simulating a drug's pharmacokinetic parameters, predicting drug concentrations over time, optimizing dosage regimens, linking concentrations with pharmacologic activity, and estimating potential toxicity.
There are three primary types of models: empirical, compartment, and physiological. Empirical models, with minimal...
There are three primary types of models: empirical, compartment, and physiological. Empirical models, with minimal...
1.9K