对人工智能聊天机器人的比较评估,以回答与脑电图相关的问题
Soraia Proença1,2, Joana Isabel Soares2,3,4, Joana Parra5
1Neurophysiology Department, Luz Hospital - Torres de Lisboa, Lisbon, Portugal.
Epileptic disorders : international epilepsy journal with videotape
|December 16, 2025
概括
大型语言模型 (LLM) 可以解释脑电图 (EEG) 概念,而ChatGPT显示出比Copilot和Gemini更高的准确性和完整性. 然而,由于潜在的不准确性和可读性问题,LLM生成的EEG信息需要进行验证.
科学领域:
- 人工智能在医学中的应用
- 神经科学和临床神经生理学
背景情况:
- 大型语言模型 (LLM) 为解释诸如脑电图 (EEG) 等复杂的医疗概念给非专家提供了潜力.
- 在专业领域,LLM的可访问性需要评估其实用性和准确性.
研究的目的:
- 为了比较三个基于LLM的聊天机器人生成的EEG相关解释的准确性,完整性和可读性:ChatGPT,Copilot和Gemini.
- 评估临床神经生理学专家评估聊天机器人的反应之间的评审者间的协议.
主要方法:
- 在10个类别中,向ChatGPT,Copilot和Gemini提出了100个EEG相关的问题.
- 六位临床神经生理学评估者 (医生,教师,技术人员) 评估了准确性 (6分级) 和完整性 (3分级) 的答案.
- 使用自动化可读性指数 (ARI) 评估可读性;统计分析包括ANOVA和ICC可靠性.
主要成果:
- 与Copilot (4.11) 和Gemini (4.16) 相比,ChatGPT的整体准确性显著更高 (4.54).
- 聊天GPT也获得了更高的完整性分数,但其响应的可读性 (ARI 17.41) 比Copilot (11.14) 和Gemini (14.16) 低.
- 对于医生和教师的准确性,评价者之间的一致性很好,但对于某些EEG类别的技术人员来说,情况很差.
结论:
- 通过LLM生成的EEG解释显示了相对较高的准确性,但含有缺陷,需要专家验证.
- 对于EEG主题,ChatGPT在准确性和完整性方面表现出色,但牺牲了可读性.
- 评审者之间协议的差异,特别是技术人员之间的差异,突出了标准化的EEG培训和评估中的潜在差距.
相关概念视频
01:16Non-equilibrium in the Cell
An important concept in studying metabolism and energy is that of chemical equilibrium. Most chemical reactions are reversible. They can proceed in both directions, releasing energy into their environment in one direction, and absorbing it from the environment in the other direction. The same is true for the chemical reactions involved in cell metabolism, such as the breaking down and building up of proteins into and from individual amino acids, respectively. Reactants within a closed system...
01:20Introduction to Cognitive Psychology
Cognitive psychology is the field of psychology dedicated to examining how people think. It attempts to explain how and why we think the way we do by studying the interactions among human thinking, emotion, creativity, language, and problem-solving, as well as other cognitive processes. Cognitive psychology studies how information is processed and manipulated in remembering, thinking, and knowing.
This field emerged in the mid-20th century, following a period dominated by behaviorism, which...
This field emerged in the mid-20th century, following a period dominated by behaviorism, which...