对于临床数据提取,QLoRA达到接近LoRA的准确性,同时需要较低的计算资源
Prabin R Shakya1, Ayush Khaneja1, Kavishwar B Wagholikar1,2
1Massachusetts General Hospital, Boston, MA USA.
medRxiv : the preprint server for health sciences
|November 24, 2025
概括
在量化大语言模型 (LLM) 上的参数高效微调 (PEFT) 保持了临床数据提取的准确性,同时显著降低了计算需求. 这使得资源有限的医疗保健团队可以使用先进的AI.
科学领域:
- 人工智能在医学中的应用
- 医疗保健中的自然语言处理 (NLP)
- 计算语言学 计算语言学
背景情况:
- 大型语言模型 (LLM) 擅长从自由文本中提取结构化数据.
- 传统的对临床任务的LLMs微调是计算上昂贵且内存密集的.
- 参数高效微调 (PEFT) 提供了一个解决方案,只更新模型权重的子集.
研究的目的:
- 评估PEFT方法是否能在临床数据提取的量化LLM上保持准确性.
- 在量子化模型上使用PEFT评估减少内存和GPU需求.
- 为了确定在硬件有限的研究小组中使用这些方法的可行性.
主要方法:
- 微调了三个Llama-3.1-8B-Instruct变体:非量子化LoRA和量子化QLoRA (8位和4位).
- 使用了ELMTEX集体 (6万份临床摘要) 与15个类别的手册注释.
- 评估模型使用原始和高级提示,测量ROUGE,BERTScore和实体级F1.
主要成果:
- 微调显著优于单独提示,LoRA提高了10-20点的指标.
- QLoRA取得了类似的结果,改善了8-14个点,仅比LoRA低2-4个点.
- 量子化大大降低了资源需求:QLoRA使用的GPU和GPU RAM的峰值比LoRA更少,尽管训练时间增加了.
结论:
- 量子化模型上的PEFT提供了一个实用的方法,用于在资源有限的环境中准确地提取临床信息.
- 这种方法大大减少了GPU数量和内存足迹,同时保持了大多数准确度的增长.
- 需要进一步的研究来验证QLoRA跨不同的LLM架构和临床数据类型.
相关概念视频
Improving Translational Accuracy
14.0K
Base complementarity between the three base pairs of mRNA codon and the tRNA anticodon is not a failsafe mechanism. Inaccuracies can range from a single mismatch to no correct base pairing at all. The free energy difference between the correct and nearly correct base pairs can be as small as 3 kcal/ mol. With complementarity being the only proofreading step, the estimated error frequency would be one wrong amino acid in every 100 amino acids incorporated. However, error frequencies observed in...
14.0K
Improving Translational Accuracy
3.5K
3.5K
Difference from Background: Limit of Detection
8.0K
The limit of detection (LOD) is the smallest amount of analyte that can be distinguished from the background noise. The LOD value corresponds to the concentration at which the analyte signal is three times larger than the standard deviation of the blank signal. Below this value, the analyte signal cannot be differentiated from the background noise. It is calculated by dividing the calibration slope by 3 times the standard deviation of the blank signals.
The LOD indicates the presence or absence...
The LOD indicates the presence or absence...
8.0K
Accuracy and Precision
13.9K
Scientists typically make repeated measurements of a quantity to ensure the quality of their findings and to evaluate both the precision and the accuracy of their results. Measurements are said to be precise if they yield very similar results when repeated in the same manner. A measurement is considered accurate if it yields a result that is very close to the true or the accepted value. Precise values agree with each other; accurate values agree with a true value. Highly accurate...
13.9K
Accuracy, limits, and approximation
1.1K
Accuracy, limits, and approximations are common in many fields, especially in engineering calculations. These concepts are imperative for ensuring that a given value is as close as possible to its true value.
Accuracy is defined as the closeness of the measured value to the true or actual value. In engineering mechanics, repeated measurements are taken during theoretical or experimental analyses to ensure that the result is precise and accurate.
The accuracy of any solution is based on the...
Accuracy is defined as the closeness of the measured value to the true or actual value. In engineering mechanics, repeated measurements are taken during theoretical or experimental analyses to ensure that the result is precise and accurate.
The accuracy of any solution is based on the...
1.1K
Extraction: Advanced Methods
1.1K
Metal ions can be separated from one another by complexation with organic ligands–the chelating agent– to form uncharged chelates. Here, the chelating agent must contain hydrophobic groups and behave as a weak acid, losing a proton to bind with the metal. Since most organic ligands used in this process are insoluble or undergo oxidation in the aqueous phase, the chelating agent is initially added to the organic phase and extracted into the aqueous phase. The metal-ligand complex is...
1.1K


