为命名实体识别任务微调大型语言模型的样本大小考虑:方法论研究研究
Zoltan P Majdik1, S Scott Graham2, Jade C Shiva Edward2
1Department of Communication, North Dakota State University, Fargo, ND, United States.
JMIR AI
|June 14, 2024
概括
适度的样本大小有效微调生物医学命名实体识别 (NER) 的大型语言模型 (LLM). 训练数据密度是关键,质量可能超过容量以获得最佳性能.
科学领域:
- 医疗信息学 医疗信息学
- 自然语言处理自然语言处理.
- 生物医学数据科学 生物医学数据科学
背景情况:
- 大型语言模型 (LLM) 为医疗信息学应用提供了巨大的潜力.
- 然而,在生物医学和卫生政策背景下微调LLM的样本大小要求方面缺乏实际数据.
研究的目的:
- 评估样本大小和精细调整LLMs的选择技术.
- 改进对利益冲突披露声明的命名实体识别 (NER).
主要方法:
- 附注 490 利益冲突披露声明以识别"人"和"组织"实体.
- 抽取了 2500 个不同大小的分层随机样本进行微调.
- 从使用这些样本的变压器 (BERT) 和生成式预训练变压器 (GPT) 模型中训练有素的双向编码器表示.
- 评估了样本大小 (句子) 和实体密度 (每句实体[EPS]) 对NER绩效 (F1得分) 的影响.
主要成果:
- 精心调整的模型实现了高的NER性能 (F1得分:0.790.96).
- 样本大小和EPS都是模型性能的显著预测因素 (P<.001).
- 确定了样本大小 (439527句) 和EPS (1.361.38) 的边际回报.
结论:
- 对生物医学NER的LLM有效微调是可以通过适度的样本大小来实现的.
- 培训数据实体密度应该与生产数据保持一致.
- 训练数据质量和模型架构的预期用途是关键因素,可能比数据量或模型大小更重要.
相关概念视频
Sample Size Calculation
3.3K
Knowledge of the sample size is the first requirement to conduct random sampling or an experiment. The sample size is the total number of units, observations, or groups (in some cases) used to get the data to estimate a population parameter. As the name suggests, the sample size is that of the sample drawn from the population and differs from the population size.
The sample size for the given experiment or sampling effort is fundamental to any study design. Sample size decides the number of...
The sample size for the given experiment or sampling effort is fundamental to any study design. Sample size decides the number of...
3.3K
Contaminants and Errors
88
Effective sample preparation is crucial for accurate and reliable laboratory analysis. During this process, two significant sources of error can arise: concentration bias from improper sample splitting and contamination caused by methods used to reduce particle size, such as grinding or homogenization. Identifying and minimizing these potential errors is crucial to ensuring the validity of the analysis.
Another key consideration is determining the appropriate number of samples required to...
Another key consideration is determining the appropriate number of samples required to...
88
Mechanistic Models: Compartment Models in Algorithms for Numerical Problem Solving
48
Mechanistic models play a crucial role in algorithms for numerical problem-solving, particularly in nonlinear mixed effects modeling (NMEM). These models aim to minimize specific objective functions by evaluating various parameter estimates, leading to the development of systematic algorithms. In some cases, linearization techniques approximate the model using linear equations.
In individual population analyses, different algorithms are employed, such as Cauchy's method, which uses a...
In individual population analyses, different algorithms are employed, such as Cauchy's method, which uses a...
48


