相关实验视频

Updated: Jul 11, 2026

Recording Mouse Ultrasonic Vocalizations to Evaluate Social Communication

Recording Mouse Ultrasonic Vocalizations to Evaluate Social Communication

Published on: June 5, 2016

超越代币:对法语大语言模型的公平评估,用于临床命名实体识别

Jamil Zaghir^1,2, Mina Bjelogrlic^1,2, Jean-Philippe Goldman^1,2

¹Division of Medical Information Sciences, Geneva University Hospitals, Geneva, Switzerland.

Studies in health technology and informatics

|August 23, 2024

概括

评估命名实体识别 (NER) 模型揭示了代币和实体级度量之间的显著差异. 与ChatGPT相比,BERT模型在高级法语实体检测方面存在局限性.

科学领域:

自然语言处理 (NLP) 是一种自然语言处理.
计算语言学计算语言学
生物医学信息学生物医学信息学

背景情况:

基于变压器的命名实体识别 (NER) 模型在各种语言和领域中表现出色.
仅仅根据代币级度量衡量NER绩效的评估可能会产生误导.
准确的实体识别在瘤学等专业领域至关重要.

研究的目的:

调查法国NER中代币级和实体级评估指标之间的差异.
评估基于BERT的模型和ChatGPT在法国瘤数据上的表现.
突出实体级评估在NER任务中的重要性.

主要方法:

微调四个法国基于BERT的模型来进行代币分类.
使用令牌级和实体级指标评估模型性能.
通过对法国瘤学报告进行快速工程来评估ChatGPT的NER能力.

主要成果:

在代币级和实体级评估结果之间观察到显著的差异.
根据使用的评估指标,BERT模型的有效性有所不同.
与BERT相比,ChatGPT在检测高级法国实体方面存在局限性.

结论:

实体级别的指标对于对NER模型进行全面和准确的评估至关重要.
当前的评估实践可能会高估模型性能,如果仅依赖于代币级度量.
需要进一步的研究来改进NER模型,特别是针对特定领域和语言的复杂实体识别.

关键词:

临床NLP是指临床NLP.医疗快递工程医疗快递工程命名实体的认可方式

更多相关视频

Foreign Accent and Forensic Speaker Identification in Voice Lineups: The Influence of Acoustic Features Based on Prosody

Foreign Accent and Forensic Speaker Identification in Voice Lineups: The Influence of Acoustic Features Based on Prosody

Published on: September 27, 2024

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness

Published on: December 6, 2024

相关概念视频

Improving Translational Accuracy

Improving Translational Accuracy

Base complementarity between the three base pairs of mRNA codon and the tRNA anticodon is not a failsafe mechanism. Inaccuracies can range from a single mismatch to no correct base pairing at all. The free energy difference between the correct and nearly correct base pairs can be as small as 3 kcal/ mol. With complementarity being the only proofreading step, the estimated error frequency would be one wrong amino acid in every 100 amino acids incorporated. However, error frequencies observed in...

Improving Translational Accuracy

Improving Translational Accuracy

Base complementarity between the three base pairs of mRNA codon and the tRNA anticodon is not a failsafe mechanism. Inaccuracies can range from a single mismatch to no correct base pairing at all. The free energy difference between the correct and nearly correct base pairs can be as small as 3 kcal/ mol. With complementarity being the only proofreading step, the estimated error frequency would be one wrong amino acid in every 100 amino acids incorporated. However, error frequencies observed in...

Components of Language

Components of Language

Language, whether spoken, signed, or written, consists of specific components: lexicon and grammar. The lexicon is the vocabulary of a language, comprising its words. Grammar is the set of rules used to convey meaning through the lexicon. For example, English grammar adds “-ed” to most verbs to indicate past tense. Words are formed by combining phonemes, which are the basic sound units of a language. Different languages have different sets of phonemes (e.g., “ah” vs. “eh”). Phonemes combine to...

Language and Cognition

Language and Cognition

Language serves as a bridge between ideas and communication, influencing how individuals perceive and interact with the world. Psychologists have long debated whether language shapes thought or vice versa. This discussion gained grip with Edward Sapir and Benjamin Lee Whorf in the 1940s, who proposed that language determines thought, a concept known as linguistic determinism. They suggested that the vocabulary and structure of a language influence how its speakers think and perceive reality.