评估大型语言模型以增强放射学专业检查:与人类表现的比较研究
Hao-Yun Liu1, Shyh-Jye Chen2, Weichung Wang3
1Department of Medical Imaging, National Taiwan University Hospital Hsin-Chu Branch, Br. No.25, Lane 442, Sec.1, Jingguo Rd., Hsinchu City 300, Taiwan ROC (H.Y.L.); National Taiwan University College of Medicine, No.1 Jen Ai Road Section 1, Taipei 100, Taiwan ROC (H.Y.L.).
Academic radiology
|May 28, 2025
概括
先进的大型语言模型 (LLM) 在放射学考试中表现有前途,GPT-4o和o1-预览表现优于人类考生. 这些人工智能工具可以帮助评估问题难度和标准化医疗检查.
科学领域:
- 医疗教育 医学教育
- 人工智能的人工智能
- 放射学 放射学是一门学科.
背景情况:
- 由于医学知识不断扩大,传统的放射学检查在准确性和相关性方面面临挑战.
- 大型语言模型 (LLM) 显示了提高医学教育和评估的潜力.
- 为放射学考试设计制定客观框架至关重要.
研究的目的:
- 为了评估LLM在放射学专业考试中的表现.
- 探索LLM在评估问题难度和推理过程中的作用.
- 研究LLMs在开发更客观,更有效的考试设计框架方面的潜力.
主要方法:
- 在放射学考试中比较了三个LLM (GPT-4o,o1-preview,GPT-3.5-turbo-1106) 与人类受试者的性能.
- 用于LLM评估的零射击条件.
- 评估准确性,歧视指数和点-二次相关性,以分析问题难度和推理.
主要成果:
- 在准确度方面,GPT-4o (88.0%) 和o1-预览 (90.9%) 显著超过了人类受试者 (76.3%).
- GPT-3.5-turbo-1106的准确性较低 (50.2%),性能变化更大.
- 高级LLM准确地识别了区分问题,反映了人类推理模式,并评估了考试难度.
结论:
- 像GPT-4o和o1-preview这样的高级LLM在放射学考试中表现出强大的解决问题的能力.
- LLM可以作为评估考试题难度的有价值工具.
- 法律学士可以协助医疗检查的标准化开发和评估.
相关概念视频
Stereotype Content Model
13.1K
The Stereotype Content Model (SCM) was first proposed by Susan Fiske and her colleagues (Fiske, Cuddy, Glick & Xu, 2002; see also Fiske, 2012 and Fiske, 2017). The SCM specifies that when someone encounters a new group, they will stereotype them based on two metrics: warmth—or that group’s perceived intent, and how likely they are to provide help or inflict harm—and competence—or their ability to carry out that objective. Depending on the warmth-competence...
13.1K
Modeling in Therapy
823
Modeling, a key technique in therapy, uses observational learning to help clients acquire and practice new skills by watching therapists demonstrate desired behaviors. This approach, rooted in Albert Bandura's concept of vicarious learning, plays a significant role in therapeutic interventions for various psychological conditions, including social anxiety, ADHD, and depression.
Participant Modeling
Participant modeling involves therapists demonstrating calm and effective behaviors in...
Participant Modeling
Participant modeling involves therapists demonstrating calm and effective behaviors in...
823
Self-Evaluation Maintenance Model
423
The Self-Evaluation Maintenance (SEM) model offers a psychological framework to understand how individuals’ self-esteem is influenced by the achievements of others, particularly those with whom they share close personal bonds. The SEM model operates when personal rather than social identity guides individuals. Central to this model is the notion that individuals have an inherent desire to preserve a favorable self-image, which is continuously shaped by interpersonal comparisons and...
423
Non-Verbal Cues
774
Non-verbal communication extends beyond gestures and facial expressions to include vocal elements known as paralanguage. Paralanguage consists of non-verbal vocal cues such as pitch, loudness, speech rate, pauses, and non-verbal vocalizations like laughter, sighs, and moans. These elements not only accompany speech but also provide critical emotional and contextual information.The Role of Paralanguage in CommunicationParalanguage adds depth to spoken language by conveying emotions and...
774


