相关实验视频

Updated: Jul 26, 2025

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness

Published on: December 6, 2024

生成型大语言模型在眼科科委会风格问题上的表现

Louis Z Cai¹, Abdulla Shaheen¹, Andrew Jin²

¹From the Bascom Palmer Eye Institute, Miami, Florida, USA (L.Z.C., A.S., J.S.Y., N.Y., C.A.).

American journal of ophthalmology

|June 20, 2023

概括

像ChatGPT-4.0和Bing Chat这样的生成人工智能模型在板式问题上显示出与人类眼科医生的准确性相比较的准确性. 然而,这些大型语言模型 (LLM) 仍然需要改进,以减少幻觉和增强推理.

科学领域:

眼科医生眼科眼科
人工智能的人工智能
医学教育医学教育

背景情况:

生成型人工智能 (AI) 模型越来越能够处理和生成类似人类的文本.
评估这些大型语言模型 (LLM) 在专业医疗领域的性能对于理解它们的潜在应用至关重要.
眼科董事会式的问题提供了医学知识和临床推理的严格测试.

研究的目的:

评估生成性AI模型在回答眼科董事会式问题的准确性和可靠性.
为了比较不同LLM (ChatGPT 3.5,ChatGPT 4.0,Bing Chat) 与人类受访者的表现.
确定优势和弱点的领域,包括推理能力和信息编造的频率.

主要方法:

一项实验性研究利用了250个眼科板式问题从基础科学和临床科学自我评估计划.
对三种LLM进行评估:Bing Chat,ChatGPT 3.5和ChatGPT 4.0.0等.
将AI性能与人类受访者的性能进行比较,分析准确性,问题复杂性和患者护理阶段.

主要成果:

人类受访者平均准确率为72.2%.
聊天GPT-4.0 (71.6%) 和Bing聊天 (71.2%) 的表现与人类相当,显著优于聊天GPT-3.5 (58.8%).
聊天GPT-4.0在工作类型问题中表现强,但在图像解释方面遇到了困难,而Bing Chat在图像解释和多步推理方面也面临挑战. 幻觉发生率各不相同,ChatGPT-3.5出现率最高 (42.4%).

结论:

像ChatGPT-4.0和Bing Chat这样的高级LLM可以在眼科董事会式评估中达到与人类专家相似的性能水平.
尽管精度相似,但人工智能反应中的幻觉和非逻辑推理的频率凸显了在医疗应用中需要进一步开发和验证.
该研究强调了人工智能在医学教育和评估中的潜力,同时强调了解决当前局限性的重要性.

更多相关视频

Evidence-based Knowledge Synthesis and Hypothesis Validation: Navigating Biomedical Knowledge Bases via Explainable AI and Agentic Systems

Evidence-based Knowledge Synthesis and Hypothesis Validation: Navigating Biomedical Knowledge Bases via Explainable AI and Agentic Systems

Published on: June 13, 2025

Application of Deep Learning-Based Medical Image Segmentation via Orbital Computed Tomography

Application of Deep Learning-Based Medical Image Segmentation via Orbital Computed Tomography

Published on: November 30, 2022

相关概念视频

Improving Translational Accuracy

Improving Translational Accuracy

Glaucoma: Overview

Glaucoma: Overview

Glaucoma is an eye condition characterized by increased intraocular pressure that damages the retina and optic nerve, leading to irreversible blindness if left untreated. The human eye has various components, including the cornea, iris, pupil, lens, and optic nerve. Aqueous humor is secreted by the epithelium of the ciliary body in the posterior chamber and flows through the trabecular meshwork and canal of Schlemm, maintaining normal intraocular pressure. The trabecular meshwork and the canal...