人工智能对常见问题产生的答案:评估GPT-4o的性能准确性和一致性
Aslinur Keles1, Ozge Gulsum Illeez1, Berkay Erbagci1
1Department of Physical Medicine and Rehabilitation, Health Science University, Fatih Sultan Mehmet Training and Research Hospital, İstanbul, Türkiye.
Archives of rheumatology
|April 23, 2025
概括
GPT-4o提供了大多数准确的coccydynia信息,但在重复查询中显示了不一致性. 虽然它对患者教育很有用,但它需要人类监督来获得专门的临床知识.
科学领域:
- 人工智能在医学中的应用
- 医疗信息学 医疗信息学
- 自然语言处理自然语言处理.
背景情况:
- 关于囊炎的患者中心问题是常见的.
- 评估人工智能产生的医疗信息的准确性和一致性对于患者安全和教育至关重要.
研究的目的:
- 评价GPT-4o对有关状腺菌的常见问题答案的准确性和一致性.
- 确定AI聊天机器人为患者提供医疗信息的可靠性.
主要方法:
- 从各种在线来源收集了以患者为中心的coccydynia问题.
- 两个物理医生对GPT-4o的反应进行了准确性和完整性的评估.
- 通过使用统计分析,评估多个查询和审查者的响应一致性.
主要成果:
- 45.7%的GPT-4o反应是正确和全面的; 49.4%是正确但不完整的.
- 没有任何答案是完全不正确的,但在查询轮之间有75%的差异.
- 面试人员的一致性很大 (kappa=0.67).
结论:
- GPT-4o 显示了准确的可克西迪尼亚信息传播的潜力.
- 响应的变化凸显了临床应用中需要人类监督的需要.
- 人工智能聊天机器人可以作为补充工具,用于对患者的教育.
相关概念视频
01:52Accuracy and Precision
Scientists typically make repeated measurements of a quantity to ensure the quality of their findings and to evaluate both the precision and the accuracy of their results. Measurements are said to be precise if they yield very similar results when repeated in the same manner. A measurement is considered accurate if it yields a result that is very close to the true or the accepted value. Precise values agree with each other; accurate values agree with a true value. Highly accurate measurements...
01:52Accuracy and Precision
Scientists typically make repeated measurements of a quantity to ensure the quality of their findings and to evaluate both the precision and the accuracy of their results. Measurements are said to be precise if they yield very similar results when repeated in the same manner. A measurement is considered accurate if it yields a result that is very close to the true or the accepted value. Precise values agree with each other; accurate values agree with a true value. Highly accurate measurements...