目前的人工智能聊天机器人对眼科科学摘要和参考文献的评估和比较
Hong-Uyen Hua1, Abdul-Hadi Kaakour1, Aleksandra Rachitskaya1
1Cole Eye Institute, Cleveland Clinic Foundation, Cleveland, Ohio.
JAMA ophthalmology
|July 27, 2023
概括
人工智能聊天机器人生成平均质量的科学摘要,但产生高比例的虚假引用. 建议在使用人工智能进行健康教育和学术写作时谨慎使用,因为可能存在不准确性.
科学领域:
- 眼科医生 眼科 眼科
- 人工智能的人工智能
- 医学教育 医学教育
背景情况:
- 基于语言学习模型的人工智能 (AI) 聊天机器人越来越多地用于学术界和患者教育.
- 潜在的缺点,如人工智能产生的内容不准确性 (幻觉),需要进行彻底的调查,特别是在科学背景下.
研究的目的:
- 为了比较眼科科学摘要和引用的质量,由人工智能聊天机器人的早期和更新版本生成.
- 评估与人工智能生成的参考相关的幻觉率.
主要方法:
- 一项横截面的比较研究使用了两种AI聊天机器人版本来生成七个眼科子专业的临床研究问题的摘要和参考.
- 摘要使用修改的DISCERN标准和绩效评分进行评估. 人工智能输出检测器评估了抽象真实性.
- 计算了无法验证的引用的幻觉率,并将聊天机器人版本进行了比较.
主要成果:
- 早期和更新的AI聊天机器人版本都产生了摘要,平均修改后的AI-DISCERN分数分别为35.9和38.1 (共50分).
- 人工智能输出检测器表明,在更新版本的检测器中,一个检测器的AI生成的抽象得分显著下降 (65.4%至10.8%,P=.01).
- 无法验证的参考文献的平均幻觉率为早期版本的33%,更新版本的29% (P=.74).
结论:
- 人工智能聊天机器人生成科学上平均质量的摘要.
- 在生成虚假引用时出现大量幻觉,因此在使用人工智能工具用于健康教育或学术目的时需要谨慎.
- 需要进一步的研究来完善AI在科学内容生成中的准确性.
更多相关视频
05:47Evidence-based Knowledge Synthesis and Hypothesis Validation: Navigating Biomedical Knowledge Bases via Explainable AI and Agentic Systems
Published on: June 13, 2025
296
06:29Comparing Objective Conjunctival Hyperemia Grading and the Ocular Surface Disease Index Score in Dry Eye Syndrome During COVID-19
Published on: May 25, 2022
2.4K
相关概念视频
01:25Glaucoma: Overview
623
Glaucoma is an eye condition characterized by increased intraocular pressure that damages the retina and optic nerve, leading to irreversible blindness if left untreated. The human eye has various components, including the cornea, iris, pupil, lens, and optic nerve. Aqueous humor is secreted by the epithelium of the ciliary body in the posterior chamber and flows through the trabecular meshwork and canal of Schlemm, maintaining normal intraocular pressure. The trabecular meshwork and the canal...
623
01:28Angle Closure Glaucoma: Treatment
565
Angle-closure glaucoma, or closed-angle glaucoma, is an eye condition where the iris bulges out and blocks the iridocorneal angle, resulting in a buildup of aqueous humor and increased intraocular pressure. Immediate medical attention is necessary due to the sudden onset of symptoms. The treatment for angle-closure glaucoma includes short-term and long-term approaches. Short-term treatment involves using eye drops like pilocarpine to lower intraocular pressure by increasing aqueous humor...
565
01:27Open Angle Glaucoma: Treatment
481
In open-angle glaucoma, the iridocorneal angle remains open, but the trabecular meshwork becomes stiff, slowing down the outflow of aqueous humor. This causes a buildup of aqueous humor in the anterior chamber, leading to a sudden increase in intraocular pressure. The treatment for open-angle glaucoma focuses on reducing the elevated intraocular pressure by either decreasing the secretion of aqueous humor or increasing its outflow.
Drugs such as carbonic anhydrase inhibitors, α2- and...
Drugs such as carbonic anhydrase inhibitors, α2- and...
481