评估大型语言模型与大型语言模型的对比 白内障和折射手术的住户:使用AAO BCSC自我评估计划进行比较分析
Avi Wallerstein1,2, Taanvee Ramnawaz2,3, Mathieu Gauvin1,2
1Department of Ophthalmology and Visual Sciences, McGill University, Montreal, QC, Canada.
Journal of cataract and refractive surgery
|October 31, 2025
概括
聊天GPT-4o在回答白内障和折射手术问题方面表现出卓越的准确性,明显优于眼科住院医生和其他大型语言模型 (LLM). 快速复杂性没有影响LLM的表现.
科学领域:
- 眼科医生 眼科 眼科
- 人工智能的人工智能
- 医学教育 医学教育
背景情况:
- 大型语言模型 (LLM) 越来越多地用于各种领域,包括医学.
- 评估LLM在眼科等专业医疗领域的准确性至关重要.
- 对于眼科医生来说,白内障和折射手术知识是必不可少的.
研究的目的:
- 评估领先的LLM在回答与白内障和折射手术有关的问题的准确性.
- 为了确定即时复杂性是否影响LLM在这个领域的表现.
- 为了比较LLM准确度与眼科住院医生的表现.
主要方法:
- 一项基于问卷的随机研究使用了BCSC自我评估计划的白内障和折射手术部分的100个问题.
- 测试了五种LLM (ChatGPT-4,ChatGPT-4o,双子座,双子座高级和副驾驶精确模式).
- 使用简单和上下文提示来评估LLM,准确度定义为正确答案的比例. 来自1983名BCSC用户的数据作为比较.
主要成果:
- 聊天GPT-4o实现了最高的准确性 (84%的简单提示,86%的复杂提示).
- 简单和复杂提示之间的性能差异对于任何LLM都没有统计学意义.
- 聊天GPT-4o是唯一显著超过眼科住院医生的模型 (平均得分为77%),而Gemini Advanced和Copilot显示出更高的性能趋势.
结论:
- 聊天GPT-4o在各种提示类型中表现出优越和一致的性能,并且明显超过了眼科住院医生.
- 在这项研究中,快速复杂性并没有显著改变LLM准确性.
- LLM,特别是ChatGPT-4o,显示出作为折射手术知识评估的有价值工具的潜力,大多数模型都超过了60%的准确度值.
更多相关视频
05:46Author Spotlight: Advancements in Refractive Surgical Correction for Presbyopia and Exploring Postoperative Visual Acuity
Published on: September 20, 2024
760
05:14Comparison of Agreement and Accuracy using Binocular Wavefront Optometer with Autorefractor and Phoropter
Published on: September 16, 2025
549
相关概念视频
Sleep Apnea
Sleep apnea is a condition where breathing stops intermittently during sleep, often leading to significant health issues. Each episode can last from 10 to 20 seconds or more and is frequently accompanied by a brief arousal from sleep. This disturbance, largely unnoticed by the individual, can lead to severe daytime fatigue. Commonly, individuals seek help after being informed by their partners about loud snoring and noticeable breathing pauses during sleep.
The condition is more prevalent among...
The condition is more prevalent among...
Cardiopulmonary Resuscitation II: ACLS Airway Management
Airway management is a key skill in emergency and critical care settings, as maintaining a clear airway is essential for adequate oxygenation and ventilation.Head Tilt-Chin Lift TechniqueThe head tilt-chin lift maneuver is an essential technique primarily used in patients without suspected cervical spine injuries. To perform this maneuver, one hand is placed on the patient’s forehead, and gentle pressure is applied backward to tilt the head. The fingertips of the other hand are positioned under...
