基于指南的PGx建议的复制大型语言模型的基准测试
Mike Zack1, Ioan Slobodchikov2, Danil Stupichev2
1PGxAI Inc., 330 E Charleston Rd, Palo Alto, CA, 94306, USA. mz@pgx.ai.
The pharmacogenomics journal
|July 27, 2025
概括
大型语言模型 (LLMs) 显示出产生药物基因组 (PGx) 建议的前景. 与一般模型相比,适应领域的LLM实现了更高的准确性和速度,使得人工智能驱动的个性化医疗更安全.
科学领域:
- 人工智能在医学中的应用
- 药物基因组学 药物基因组学
- 临床决策支持 临床决策支持
背景情况:
- 大型语言模型 (LLM) 越来越多地被用于临床应用.
- 准确的药物基因组 (PGx) 建议对于个性化医学至关重要.
- 现有的LLM可能缺乏像CPIC这样复杂的临床指南的特异性.
研究的目的:
- 评估LLMs在制造药基因组学 (PGx) 建议中的临床准确性.
- 将通用LLM与PGx的域调整模型的性能进行比较.
- 建立一个评估PGx特定LLM绩效的框架.
主要方法:
- 使用了599个精心策划的基因-药物-表型场景的基准.
- 评估了五个领先的LLM,包括GPT-4o和微调的LLaMA变体.
- 一个新的语义评估框架 (LLM分数),经过专家审查验证,与词汇指标一起使用.
主要成果:
- 一般用途的LLM经常产生不完整或不安全的PGx建议.
- 一个适应领域的LLM获得了0.92的高LLM得分,表明了卓越的表现.
- 与其他模型相比,适应域的模型表现出明显更快的推理速度.
结论:
- 微调和结构化提示对于开发准确的PGxLLMs至关重要,仅仅超越模型规模.
- 针对特定领域的适应对于药物基因组学中安全有效的AI至关重要.
- 这项研究验证了评估PGxLLM的框架,并支持人工智能驱动的个性化医学.
相关概念视频
Pharmacokinetic Models: Comparison and Selection Criterion
150
Physiological and compartmental models are valuable tools used in studying biological systems. These models rely on differential equations to maintain mass balance within the system, ensuring an accurate representation of the dynamic processes at play.
Physiological models take a detailed approach by considering specific molecular processes. They can predict drug distribution, metabolism, and elimination changes, providing a comprehensive understanding of how drugs interact with the body.
Physiological models take a detailed approach by considering specific molecular processes. They can predict drug distribution, metabolism, and elimination changes, providing a comprehensive understanding of how drugs interact with the body.
150
Analysis of Population Pharmacokinetic Data
387
Analysis of population pharmacokinetic data involves studying the behavior of drugs within diverse populations to understand their pharmacokinetic parameters. Traditional pharmacokinetic methods typically involve collecting samples from a few individuals and estimating these parameters. While these methods are commonly used, they have limitations in capturing the variability in drug response among individuals or heterogeneous populations. Population pharmacokinetics is employed to address these...
387
Pharmacokinetic Models: Overview
1.1K
Pharmacokinetic models utilize mathematical analysis to achieve a detailed quantitative understanding of a drug's life cycle within the body. They are instrumental in simulating a drug's pharmacokinetic parameters, predicting drug concentrations over time, optimizing dosage regimens, linking concentrations with pharmacologic activity, and estimating potential toxicity.
There are three primary types of models: empirical, compartment, and physiological. Empirical models, with minimal...
There are three primary types of models: empirical, compartment, and physiological. Empirical models, with minimal...
1.1K
Comparing Copy Number Variations and SNPs
17.9K
Sequencing of the human genome has opened up several best-kept secrets of the genome. Scientists have identified thousands of genome variations that exist within a population. These variations can be a single nucleotide or a larger chromosomal variation.
Copy number variations or CNVs are the structural variations that cover more than 1kb of DNA sequence. The single nucleotide polymorphism (SNP), on the other hand, is a single nucleotide change or a point mutation that is found in more than 1%...
Copy number variations or CNVs are the structural variations that cover more than 1kb of DNA sequence. The single nucleotide polymorphism (SNP), on the other hand, is a single nucleotide change or a point mutation that is found in more than 1%...
17.9K
Improving Translational Accuracy
11.9K
Base complementarity between the three base pairs of mRNA codon and the tRNA anticodon is not a failsafe mechanism. Inaccuracies can range from a single mismatch to no correct base pairing at all. The free energy difference between the correct and nearly correct base pairs can be as small as 3 kcal/ mol. With complementarity being the only proofreading step, the estimated error frequency would be one wrong amino acid in every 100 amino acids incorporated. However, error frequencies observed in...
11.9K
Analysis Methods of Pharmacokinetic Data: Model and Model-Independent Approaches
228
Drug disposition in the body is a complex process and can be studied using two major approaches: the model and the model-independent approaches.
The model approach uses mathematical models to describe changes in drug concentration over time. Pharmacokinetic models help characterize drug behavior in patients, predict drug concentration in the body fluids, calculate optimum dosage regimens, and evaluate the risk of toxicity. However, ensuring that the model fits the experimental data accurately...
The model approach uses mathematical models to describe changes in drug concentration over time. Pharmacokinetic models help characterize drug behavior in patients, predict drug concentration in the body fluids, calculate optimum dosage regimens, and evaluate the risk of toxicity. However, ensuring that the model fits the experimental data accurately...
228


