心理测量属性和可检测性GPT-4o生成的多选题与人类作者在成像专业的项目相比
Philipp Linde1, Florian Fichter2, Markus Dietlein3
1Department of Radiation Oncology, Cyberknife and Radiation Therapy, Faculty of Medicine and University Hospital of Cologne, University of Cologne, Cologne, Germany. philipp.linde@uk-koeln.de.
NPJ digital medicine
|January 8, 2026
概括
大型语言模型 (LLM) 可以生成医学教育评估问题,相当于专家撰写的问题. GPT-4o生成的多选题 (MCQ) 显示了类似的难度和歧视,并且无法识别为由考生创建的AI.
科学领域:
- 医疗教育 技术 技术 医学教育
- 医疗保健中的人工智能
- 心理测量评估 心理测量评估
背景情况:
- 大型语言模型 (LLM) 提供了扩展医学教育评估的潜力.
- 人工智能产生的评估项目的心理测量等价性和检测性存在担忧.
研究的目的:
- 评估GPT-4o生成的多选择题 (MCQ) 与医学教育中人类撰写的MCQ相比的心理等价性.
- 评估医学学生和医生对人工智能生成的MCQ的检测能力.
主要方法:
- 一个盲目的,主体内比较24个GPT-4o生成和24个人类撰写的MCQ在辐射瘤学,放射学和核医学.
- 82名医学学生和46名医生完成了48项形式模拟考试.
- 分析了项目难度,歧视以及参与者识别项目来源的能力.
主要成果:
- 在项目难度 (LLM:0.67对人类:0.65) 或歧视 (LLM:0.29对人类:0.27) 方面没有发现显著差异.
- 参与者无法识别超过机会水平 (0.50) 的MCQs的来源.
- 专家评级显示,对于适当性和教学质量 (ICC=0.07-0.18) 的评级者之间的一致性很低.
结论:
- 由GPT-4o生成的MCQ显示了与医学教育中专家撰写的项目相比较的心理测量特性.
- 在这项研究中,受试者无法可靠地检测到人工智能生成的评估项目.
- 人在循环中的方法为扩大医学教育中的形成性评估内容提供了一条可行的途径,需要明确的监督和透明度政策.
相关概念视频
01:22Self-Report Tests of Personality
757
Self-report inventories are objective personality assessments that use multiple-choice items or numbered scales, typically ranging from 1 (strongly disagree) to 5 (strongly agree). They are often called Likert scales after Rensis Likert. These inventories are widely used due to their ease of administration and cost-effectiveness. One of the most prominent examples is the Minnesota Multiphasic Personality Inventory (MMPI), initially developed in the 1940s to assess abnormal personality traits.
757
01:24Magnetic Resonance Imaging
9.0K
Magnetic resonance imaging (MRI) is a noninvasive medical imaging technique based on a phenomenon of nuclear physics discovered in the 1930s, in which matter exposed to magnetic fields and radio waves was found to emit radio signals. In 1970, a physician and researcher named Raymond Damadian noticed that malignant (cancerous) tissue gave off different signals than normal body tissue. He applied for a patent for the first MRI scanning device in clinical use by the early 1980s. The early MRI...
9.0K
01:29Positron Emission Tomography
6.9K
Positron emission tomography (PET) is a medical imaging technique involving radiopharmaceuticals — substances that emit short-lived radiation. Although the first PET scanner was introduced in 1961, it took 15 more years before radiopharmaceuticals were combined with the technique and revolutionized its potential.
One of the main requirements of a PET scan is a positron-emitting radioisotope, which is produced in a cyclotron and then attached to a substance used by the part of the body...
One of the main requirements of a PET scan is a positron-emitting radioisotope, which is produced in a cyclotron and then attached to a substance used by the part of the body...
6.9K
01:14Brain Imaging
661
Brain imaging technologies provide critical insights into both the structure and function of the human brain, enabling medical professionals and researchers to diagnose, study, and treat neurological disorders or psychiatric disorders more effectively.
These technologies include computerized axial tomography (CAT or CT scans), positron-emission tomography (PET scans), magnetic resonance imaging (MRI), functional magnetic resonance imaging (fMRI), and Transcranial Magnetic...
These technologies include computerized axial tomography (CAT or CT scans), positron-emission tomography (PET scans), magnetic resonance imaging (MRI), functional magnetic resonance imaging (fMRI), and Transcranial Magnetic...
661