使用大型语言模型评估居民反:我们是否缺少核心能力?
Syed Ameen Ahmad1, Maria Armache2, Danielle R Trakimas2
1Johns Hopkins University School of Medicine, Baltimore, Maryland, USA.
The Laryngoscope
|June 27, 2025
概括
大型语言模型可以有效地评估外科住院医生的反. 不同的评估工具产生不同的反质量,突出了对居民教育多式联络方法的需要.
科学领域:
- 医学教育 医学教育
- 医疗保健中的人工智能
- 手术培训评估的评估
背景情况:
- 有效的叙事反对于外科住院医生的发展至关重要.
- 目前的评估方法,包括基于工作场所的评估 (SIMPL-OR),技术技能目标结构评估 (OSATS) 和轮值结束 (EOR) 评估,提供不同质量的反.
- 优化反需要了解每个评估模式的优点和弱点.
研究的目的:
- 利用大型语言模型 (LLM) 来分析给外科住院医生提供的叙事反的内容和质量.
- 通过三种不同的评估格式进行反比较:SIMPL-OR,OSATS和EOR评估.
- 评估LLM与教师反评估的一致性.
主要方法:
- 一个单一机构从2017-2021年对叙事反的回顾性分析.
- 两个教师对60个反条目 (20个格式) 的初步评估,根据具体标准 (鼓励,纠正,特定和核心能力).
- 通过对教师评价来验证ChatGPT4o的表现,然后对其余776个反条目进行分析.
主要成果:
- 在评估反方面,ChatGPT与教师表现出很高的一致性 (90%, κ=0.94).
- 针对能力的显著差异被观察到:患者护理在SIMPL-OR中最高 (97%),而专业性在EOR中最高 (40%).
- SIMPL-OR提供了最"纠正" (71%) 和"特定" (97%) 的反,而OSATS和EOR显示了较低的特异性和纠正反率.
结论:
- 不同的反工具会产生不同内容和质量的反.
- 对反的多式联络方法对于全面的居民评估和发展至关重要.
- 法律法规显示,作为一种分析和潜在提高教育反质量的工具,它是有前途的.
更多相关视频
相关概念视频
Language Development
460
Children master language quickly and with relative ease, supported by both biological predisposition and reinforcement. B. F. Skinner (1957) proposed that language is learned through reinforcement, while Noam Chomsky (1965) argued that language acquisition mechanisms are biologically determined.
The critical period for language acquisition suggests that the ability to acquire language is at its peak early in life. As people age, this proficiency decreases. Language development begins very...
The critical period for language acquisition suggests that the ability to acquire language is at its peak early in life. As people age, this proficiency decreases. Language development begins very...
460
Language and Cognition
456
Language serves as a bridge between ideas and communication, influencing how individuals perceive and interact with the world. Psychologists have long debated whether language shapes thought or vice versa. This discussion gained grip with Edward Sapir and Benjamin Lee Whorf in the 1940s, who proposed that language determines thought, a concept known as linguistic determinism. They suggested that the vocabulary and structure of a language influence how its speakers think and perceive reality.
456
Language
430
Language is a unique communication system that uses words and systematic rules to organize and transmit information. Unlike other forms of communication, which may involve postures, movements, odors, or vocalizations, language relies on symbols and grammar. This makes human communication distinct from that of other species, who also communicate but do not use language in the same way humans do.
Corballis and Suddendorf (2007) and Tomasello and Rakoczy (2003) highlight the role of language in...
Corballis and Suddendorf (2007) and Tomasello and Rakoczy (2003) highlight the role of language in...
430
Components of Language
410
Language, whether spoken, signed, or written, consists of specific components: lexicon and grammar. The lexicon is the vocabulary of a language, comprising its words. Grammar is the set of rules used to convey meaning through the lexicon. For example, English grammar adds “-ed” to most verbs to indicate past tense. Words are formed by combining phonemes, which are the basic sound units of a language. Different languages have different sets of phonemes (e.g., “ah” vs.
410
Nursing Evaluation
3.6K
The evaluation stage signals the end of the nursing process. The nurse gathers evaluative data to assess whether or not the patient has attained the expected results. Whereas the nurse collects data in the nursing assessment to identify the patient's health concerns, the evaluation stage data determines if the indicated health issues are resolved. Evaluative data collection includes two sections: the data acquired to evaluate patient outcomes and the time criteria for data collection.
3.6K
Higher Mental Functions of the Brain: Language
1.0K
Language is a system of communication that allows the expression of thoughts, ideas, and feelings. The brain processes language in both hemispheres.
Language formation and comprehension take place in the dominant hemisphere. The dominant hemisphere is responsible for understanding the meaning of spoken, written, or sign language, as well as the ability to communicate. For most people, the left hemisphere is the dominant one. The right hemisphere, then, gives tone and emotional context to the...
Language formation and comprehension take place in the dominant hemisphere. The dominant hemisphere is responsible for understanding the meaning of spoken, written, or sign language, as well as the ability to communicate. For most people, the left hemisphere is the dominant one. The right hemisphere, then, gives tone and emotional context to the...
1.0K


