DeepSeek-R1用于放射科住院考试的自动评分:一个协议和测试-重新测试可靠性研究
Shanshan Niu1, Xiaobin Liu1, Lanfang Huang1
1Department of Radiology, Fifth Affiliated Hospital of Sun Yat-sen University, Zhuhai, Guangdong, China.
BMC medical education
|November 11, 2025
概括
DeepSeek-R1在评分放射科医生住院报告方面表现与放射科医生相似. 这种人工智能工具显示了协助医学教育评估和减少教师工作量而不会影响评估质量的潜力.
科学领域:
- 医学教育 医学教育
- 医疗保健中的人工智能
- 放射学 放射学是一门学科.
背景情况:
- 越来越多地探索自动评分系统,以提高医学教育的效率.
- 放射科住院医生的评估需要一致可靠的评估方法.
研究的目的:
- 评估DeepSeek-R1的可行性和性能,用于自动评分放射科医生住院检查.
- 为了比较DeepSeek-R1的得分准确度与人类放射科医生.
主要方法:
- 一项涉及504份第三年住院医生的放射学报告的横截面研究.
- 由两个放射科医生 (放射科医生A,放射科医生B) 和DeepSeek-R1.1进行的独立评估.
- 通过使用类内相关系数 (ICC) 分析了间评分器和测试重试的可靠性.
主要成果:
- DeepSeek-R1与放射科医生A (ICC=0.879) 和放射科医生B (ICC=0.820) 取得了较高的测定器间可靠性.
- DeepSeek-R1显示出强大的测试重复测试可靠性 (ICC=0.922),与放射科医生A (ICC=0.952) 相比.
- 重新测试的DeepSeek-R1和放射科医生A之间的ICC为0.885.
结论:
- 在评估放射科医生住院报告方面,DeepSeek-R1的性能与经验丰富的放射科医生相美.
- 将DeepSeek-R1集成到医学教育中可以帮助评估任务.
- 使用像DeepSeek-R1这样的AI可能有助于减轻教师的工作量,同时保持高的评估标准.
相关概念视频
Reliability and Validity
14.4K
Reliability and validity are two important considerations that must be made with any type of data collection. Reliability refers to the ability to consistently produce a given result. In the context of psychological research, this would mean that any instruments or tools used to collect data do so in consistent, reproducible ways.
14.4K
Radiological Investigation I: X-ray and CT
1.6K
Radiological investigations, including X-rays and computed tomography (CT) scans, are critical for diagnosing and evaluating various medical conditions. These imaging techniques provide valuable insights into the body's internal structures, aiding in the detection of abnormalities, assessment of disease progression, and development of treatment strategies. This article delves into two primary radiological investigations, chest X-rays and CT scans, outlining their purpose, procedures, and...
1.6K
Radiological Investigation II: MRI and Ventilation Perfusion Scan
929
Description
Magnetic Resonance Imaging (MRI) and Ventilation Perfusion Scans are two radiological investigations that offer detailed diagnostic images of the body, particularly lung structures.
MRI
MRI uses magnetic fields and radiofrequency signals to distinguish between normal and abnormal tissues. This technology provides a more detailed diagnostic image than CT scans, enabling it to characterize pulmonary nodules, stage bronchogenic carcinoma, and evaluate inflammatory activity in...
Magnetic Resonance Imaging (MRI) and Ventilation Perfusion Scans are two radiological investigations that offer detailed diagnostic images of the body, particularly lung structures.
MRI
MRI uses magnetic fields and radiofrequency signals to distinguish between normal and abnormal tissues. This technology provides a more detailed diagnostic image than CT scans, enabling it to characterize pulmonary nodules, stage bronchogenic carcinoma, and evaluate inflammatory activity in...
929
Imaging Studies for Cardiovascular System VI: Calcium -Scoring CT
905
Calcium-Scoring CT ScanA calcium-scoring CT scan, also known as coronary artery calcium (CAC) scan, detects calcium deposits in the coronary arteries. This test assesses the risk of coronary artery disease (CAD), which can lead to cardiovascular events such as angina, heart failure, and sudden cardiac arrest.A calcium-scoring CT scan is generally recommended for individuals at intermediate risk of CAD without symptoms. It includes:Men aged 40-75 and women aged 50-75: Especially those with a...
905


