在培训期间,用大型语言模型增强可解释模型
Chandan Singh1, Armin Askari2, Rich Caruana3
1Microsoft Research, Redmond, WA, USA. chansingh@microsoft.com.
Nature communications
|November 30, 2023
概括
Aug模型利用大语言模型 (LLM) 进行高效和可解释的预测. 这个框架为推理提供了显著的速度和内存改进,使人工智能更容易获得.
科学领域:
- 人工智能的人工智能
- 自然语言处理自然语言处理.
- 计算神经科学是一种神经科学.
背景情况:
- 大型语言模型 (LLM) 显示出高性能,但缺乏可解释性和效率.
- 高风险领域和计算有限的设置需要可解释和高效的AI模型.
研究的目的:
- 提出Aug模型,一个框架,以创建高效和可解释的预测模型,利用LLMs的知识.
- 允许LLM知识转移仅用于推断,确保透明度和速度.
主要方法:
- 开发了Aug模型框架,在培训过程中使用LLM,但不推断.
- 在NLP中实例化的Aug模型:Aug-Linear (线性模型+LLM嵌入) 和Aug-Tree (决策树+LLM功能扩展).
- 应用Aug模型对文本分类和自然语言fMRI数据分析.
主要成果:
- 与LLMs相比,Aug模型显著提高了推断速度和内存效率 (超过1000倍).
- 在文本分类中,Aug-Linear和Aug-Tree的表现优于非增强式可解释模型.
- 高线性,具有1万倍较少的参数,超越了60亿参数GPT-J模型,同时保持透明.
结论:
- 在资源有限的环境中,Aug模型为可解释和高效的AI提供了可行的解决方案.
- 该框架成功地将LLM功能转移到更小,透明的模型中.
- 增强模型从复杂的数据中提供了有价值的科学解释,例如fMRI.
- 这种方法使先进的人工智能能力在各种科学领域的使用得到了民主化.
更多相关视频
05:47Evidence-based Knowledge Synthesis and Hypothesis Validation: Navigating Biomedical Knowledge Bases via Explainable AI and Agentic Systems
Published on: June 13, 2025
232
09:09Foreign Accent and Forensic Speaker Identification in Voice Lineups: The Influence of Acoustic Features Based on Prosody
Published on: September 27, 2024
460
相关概念视频
Improving Translational Accuracy
11.1K
Base complementarity between the three base pairs of mRNA codon and the tRNA anticodon is not a failsafe mechanism. Inaccuracies can range from a single mismatch to no correct base pairing at all. The free energy difference between the correct and nearly correct base pairs can be as small as 3 kcal/ mol. With complementarity being the only proofreading step, the estimated error frequency would be one wrong amino acid in every 100 amino acids incorporated. However, error frequencies observed in...
11.1K
Language and Cognition
352
Language serves as a bridge between ideas and communication, influencing how individuals perceive and interact with the world. Psychologists have long debated whether language shapes thought or vice versa. This discussion gained grip with Edward Sapir and Benjamin Lee Whorf in the 1940s, who proposed that language determines thought, a concept known as linguistic determinism. They suggested that the vocabulary and structure of a language influence how its speakers think and perceive reality.
352
Per-Unit Sequence Models
75
An ideal Y-Y transformer, grounded through neutral impedances, displays per-unit sequence networks akin to those of a single-phase ideal transformer when subjected to balanced positive- or negative-sequence currents. These currents do not produce neutral currents, and their associated voltage drops.
Zero-sequence currents, which are identical in magnitude and phase, generate a neutral current, resulting in voltage drops across the neutral impedance and the low-voltage winding. If the...
Zero-sequence currents, which are identical in magnitude and phase, generate a neutral current, resulting in voltage drops across the neutral impedance and the low-voltage winding. If the...
75
Model Approaches for Pharmacokinetic Data: Distributed Parameter Models
72
Pharmacokinetic models are mathematical constructs that represent and predict the time course of drug concentrations in the body, providing meaningful pharmacokinetic parameters. These models are categorized into compartment, physiological, and distributed parameter models.
The distributed parameter models are specifically designed to account for variations and differences in some drug classes. This model is particularly useful for assessing regional concentrations of anticancer or...
The distributed parameter models are specifically designed to account for variations and differences in some drug classes. This model is particularly useful for assessing regional concentrations of anticancer or...
72
Generalization, Discrimination, and Extinction
572
Generalization, discrimination, and extinction are key concepts in operant conditioning that influence how behaviors are learned and maintained.
Generalization occurs when a behavior reinforced in one context is performed in similar situations. For instance, a student who studies diligently for calculus and receives excellent grades might apply the same study habits to psychology and history, expecting similar results. Generalization shows how learning in one setting can influence behavior in...
Generalization occurs when a behavior reinforced in one context is performed in similar situations. For instance, a student who studies diligently for calculus and receives excellent grades might apply the same study habits to psychology and history, expecting similar results. Generalization shows how learning in one setting can influence behavior in...
572
Multicompartment Models: Overview
150
Multicompartment models are mathematical constructs that depict how drugs are distributed and eliminated within the body. They segment the body into several compartments, symbolizing various physiological or anatomical areas connected through drug transfer processes such as absorption, metabolism, distribution, and elimination.
These models offer a more comprehensive representation of drug behavior in the body than one-compartment models. They accommodate the complexity of drug distribution,...
These models offer a more comprehensive representation of drug behavior in the body than one-compartment models. They accommodate the complexity of drug distribution,...
150
