微调大型语言模型以增强研究生医学教育中的程序评估
Gregory J Booth1, Thomas Hauert1, Mike Mynes1
1The following authors are in both the Department of Anesthesiology, Uniformed Services University, Bethesda, MD, and Department of Anesthesiology and Pain Medicine, Naval Medical Center Portsmouth, Portsmouth, VA: Gregory J. Booth is an Associate Professor at Uniformed Services University and Program Director, Anesthesiology Residency at Naval Medical Center Portsmouth; Mike Mynes and Elizabeth Slama are Assistant Professors at Uniformed Services University and Staff Anesthesiologists at Naval Medical Center Portsmouth; Jeffrey Moore is an Assistant Professor at Uniformed Services University and Program Director, Pain Medicine Fellowship, and Associate Designated Institutional Official at Naval Medical Center Portsmouth. Thomas Hauert is an Anesthesiology Resident Physician at Naval Medical Center Portsmouth, Portsmouth, VA. Ashton Goldman is an Associate Professor at Uniformed Services University, Bethesda, MD, and a Staff Orthopedic Surgeon at the Department of Orthopedic Surgery and Sports Medicine at Naval Medical Center Portsmouth, Portsmouth, VA. John Hodgson is an Associate Professor and Program Director, Anesthesiology Residency at University of South Florida, Tampa, FL.
测试了大型语言模型 (LLM) 来对医学教育反进行分类. 像BERT-mini这样的较小模型的性能与较大的模型和FastText相提并论,从而提高了效率.
科学领域:
- 自然语言处理自然语言处理.
- 机器学习在医学教育中的应用
背景情况:
- 医学教育中的叙事反对于学员发展至关重要.
- 将反分类到医学研究生教育认证委员会 (ACGME) 的分管部门是具有挑战性的.
- 大型语言模型 (LLM) 提供了自动反分析的潜力.
研究的目的:
- 探索LLM复杂性和性能之间的权衡,在分类ACGME子能力方面.
- 将各种基于变压器的LLM与FastText模型的性能进行比较.
主要方法:
- 在10218条反评论中微调了几种基于变压器的LLM (BERT-基础,BERT-中等,BERT-小,BERT-迷你,BERT-小,SciBERT).
- 使用F1分数和接收器操作特征曲线 (AUC) 下的面积来比较LLM性能.
- 评估模型与之前训练的FastText模型相比.
主要成果:
- 没有基于变压器的LLM比FastText模型表现更好.
- BERT-tiny的表现比FastText的表现更糟糕.
- 伯特迷你实现了与FastText相似的性能,但比FastText小了94%.
- 对于BERT-mini和FastText,AUC得分相似,只有少数例外.
结论:
- 复杂的LLM并没有为此任务提供比更简单的模型更高的性能.
- 像BERT-mini这样的更小,更高效的LLM可以实现可比的结果,使其能够在个人设备上部署.
- 这项研究为在医学研究生教育中整合LLM提供了最佳实践信息.
相关概念视频
Mechanistic Models: Compartment Models in Algorithms for Numerical Problem Solving
In individual population analyses, different algorithms are employed, such as Cauchy's method, which uses a...
Improving Translational Accuracy


