探测药物不良反应的隐藏状态的大型语言模型知识知识
Jacob Berkowitz1, Davy Weissenbacher1, Apoorva Srinivasan1
1Department of Computational Biomedicine, Cedars-Sinai Medical Center, Los Angeles, California, USA.
medRxiv : the preprint server for health sciences
|February 24, 2025
概括
稀疏的自动编码器 (SAEs) 使大语言模型 (LLMs) 的内部知识可解释. 这种方法有效地识别药物不良反应 (ADRs) 高精度,使透明和高效的生物医学应用.
科学领域:
- 人工智能的人工智能
- 计算生物学 计算生物学
- 药物监督 药物监督 药物监督
背景情况:
- 大型语言模型 (LLM) 拥有丰富的知识,但缺乏可解释性.
- 稀疏自动编码器 (SAEs) 提供了一种方法,将LLM嵌入式线性化为可理解的功能.
- 可解释性对于人工智能的生物医学应用至关重要.
研究的目的:
- 评估Gemma Scope SAE用于解释关于药物不良反应 (ADR) 的LLM知识.
- 评估SAE衍生特征对预测ADR可能性的有用性.
- 将SAE特征上的线性分类器的性能与原始嵌入式的神经网络的性能进行比较.
主要方法:
- 将Gemma2-9b-it中的药物名称的隐藏状态嵌入式转换为可解释的SAE特征.
- 在SAE特征上训练了一种线性分类器,以预测ADR可能性.
- 使用AUC-ROC指标对基准进行分类器性能评估.
主要成果:
- SAE特征实现了对ADR的高预测性能:0.957 (急性损伤),0.902 (急性肝损伤),0.954 (急性心肌梗塞) 和0.963 (胃肠道出血).
- 在SAE输出上的线性分类器与原始嵌入上的神经网络相比没有显著的性能差异.
- 在SAE重建过程中观察到最小的信息丢失.
结论:
- 从SAE衍生的表示保留了基本的LLM信息,同时降低了模型的复杂性.
- 这种方法促进了透明和计算效率高的生物医学AI应用.
- 基于SAE的方法可以合成学习的生物医学知识,用于下游任务,如药物监测.
相关概念视频
Pharmacovigilance
764
Post-marketing surveillance is a critical component of pharmaceutical regulation, often uncovering unanticipated adverse drug reactions (ADRs) once a drug is widely used over an extended period.
This process, termed pharmacovigilance, aims to detect, evaluate, and minimize harmful effects related to medication use. The data collection for pharmacovigilance depends on spontaneous reporting systems, where healthcare professionals or patients voluntarily report suspected ADRs.
In some cases, there...
This process, termed pharmacovigilance, aims to detect, evaluate, and minimize harmful effects related to medication use. The data collection for pharmacovigilance depends on spontaneous reporting systems, where healthcare professionals or patients voluntarily report suspected ADRs.
In some cases, there...
764
Drug-Receptor Interactions
4.8K
Drug-receptor interaction describes the binding of receptors by drugs, but not all drug-receptor interactions result in activation and tissue response. For instance, the binding of agonists activates the receptor to generate a cellular reaction, while antagonists bind to receptors without causing their activation.
Several parameters, such as the drug's affinity for its receptor and its efficacy, which is its ability to activate the receptor, determine the drug's effect on the tissue....
Several parameters, such as the drug's affinity for its receptor and its efficacy, which is its ability to activate the receptor, determine the drug's effect on the tissue....
4.8K
Drug-Receptor Interaction: Antagonist
2.6K
An antagonist is a drug that binds strongly to a receptor without activating it. An antagonist prevents other molecules, such as neurotransmitters or hormones, from binding to the receptor and triggering a cellular response. Such interaction effectively hinders the normal physiological processes mediated by the receptor, resulting in various pharmacological effects depending on the specific receptor targeted.
Antagonists can be classified as competitive or noncompetitive based on their...
Antagonists can be classified as competitive or noncompetitive based on their...
2.6K
Drug Discovery: Overview
7.4K
Drug discovery is a multifaceted process involving extensive screening, testing, and optimization of lead compounds to identify potential new drugs for therapeutic use. It combines several approaches, including screening large numbers of natural products, chemical modification of known active molecules, identification of new drug targets, and rational design based on biological mechanisms and drug-receptor structure. These approaches are carried out in both academic research laboratories and...
7.4K
The Two-State Receptor Model
1.9K
The two-state receptor model explains a drug's interaction with receptors, such as G protein-coupled receptors and ligand-gated ion channels, to induce or inhibit a biological response. When no natural ligands are present, a receptor exists in an equilibrium of inactive (Ri) and active (Ra) conformations. The inactive form does not produce a response, while the active form generates a basal effect known as constitutive activity.
The binding affinity of a drug determines its interaction with...
The binding affinity of a drug determines its interaction with...
1.9K
Allergic Drug Reactions
805
Allergic reactions related to drugs are hypersensitivity responses driven by the immune system and bear no connection to the drug's therapeutic action. While drugs in isolation do not trigger an immune response, they can interact with endogenous proteins to form antigens. These antigens stimulate lymphocytes to produce antibodies. IgE-type antibodies attach themselves to mast cells. Upon subsequent exposure to the same stimulus, the antigen-antibody interaction is initiated, unleashing...
805


