针对意大利医学文本分类的大型语言模型的基准测试:生成模型是最好的选择吗?
Livia Lilli1,2, Stefano Patarnello1, Carlotta Masciocchi1
1Fondazione Policlinico Universitario Agostino Gemelli IRCCS, Rome, Italy.
Studies in health technology and informatics
|October 3, 2025
概括
这项研究对自然语言处理模型进行了基准测试,以对乳腺癌报告中的转移进行分类. 精心调整的BERT模型表现最好,即使资源有限,也显示出临床应用的潜力.
科学领域:
- 医疗信息学 医疗信息学
- 自然语言处理自然语言处理.
- 计算语言学 计算语言学
背景情况:
- 从临床报告中提取信息对于患者护理和研究至关重要.
- 自然语言处理 (NLP) 技术,包括BERT和生成的大型语言模型 (LLM),越来越多地用于此目的.
- 在识别对文本分类任务最有效的NLP方法方面仍然存在挑战,这些方法受到模型架构,数据和特定领域因素的影响.
研究的目的:
- 将生成的LLM与基于BERT的模型进行基准比较,用于在意大利乳腺癌临床报告中对转移进行分类.
- 为了在零射击学习环境中比较微调的BERT模型和生成的LLM的性能.
- 评估这些模型对现实世界的临床应用的可行性,考虑到计算限制.
主要方法:
- 在结构化生成框架内使用生成的LLM进行了比较分析.
- 基于BERT的模型已经为转移分类的具体任务进行了微调.
- 用F1分数和曲线下的面积 (AUC) 等指标评估绩效,包括BERT模型的零射击学习评估.
主要成果:
- 微调的BERT模型显示出最平衡的性能,F1得分为0.884,AUC为0.720.
- 生成型LLM显示出有希望的结果,表明进一步优化和适应的潜力.
- 基于BERT的模型和生成的LLM都在低计算设置中被证明是可行的.
结论:
- 精心调整的BERT模型为临床文本中的转移分类提供了强大的解决方案.
- 生成型的LLM提供了一个有希望的替代方案,在医学文本分类方面还有改进的余地.
- 这些NLP模型可用于现实世界的临床应用,包括具有有限计算资源的临床应用.
相关概念视频
02:07Improving Translational Accuracy
3.5K
3.5K
02:07Improving Translational Accuracy
14.1K
Base complementarity between the three base pairs of mRNA codon and the tRNA anticodon is not a failsafe mechanism. Inaccuracies can range from a single mismatch to no correct base pairing at all. The free energy difference between the correct and nearly correct base pairs can be as small as 3 kcal/ mol. With complementarity being the only proofreading step, the estimated error frequency would be one wrong amino acid in every 100 amino acids incorporated. However, error frequencies observed in...
14.1K
01:17Classification of Illness
8.6K
The meaning of illness is individualized to each person who experiences an alteration in health. In contrast, disease is a medical term indicating a pathological change in the structure and function of the body or mind. It is a condition that has specific symptoms and boundaries.
An illness is a response to a disease in which the person's level of functioning is changed compared with a previous level. The general classification of illness includes acute and chronic.
Acute illness is severe...
An illness is a response to a disease in which the person's level of functioning is changed compared with a previous level. The general classification of illness includes acute and chronic.
Acute illness is severe...
8.6K
01:29Aggregates Classification
966
Aggregate classification is generally based on its size, petrographic characteristics, weight, and source. Size classification ranges from coarse to fine aggregates, defined by the size of the particles. Coarse aggregates are particles that do not pass through ASTM sieve No. 4, and aggregates that pass through the sieve are fine aggregates.
Petrographic classification groups aggregates based on common mineralogical characteristics. Some of the common mineral groups found in aggregates are...
Petrographic classification groups aggregates based on common mineralogical characteristics. Some of the common mineral groups found in aggregates are...
966