基准AI聊天机器人:评估它们在识别被劫持的医学期刊的准确性
Mihály Hegedűs1,2, Mehdi Dadkhah3, Lóránt Dénes Dávid3,4,5,6
1Department of Finance and Accounting, Tomori Pál College, Budapest, Hungary.
Diagnosis (Berlin, Germany)
|May 27, 2025
概括
人工智能 (AI) 聊天机器人可以提供被劫持期刊的一般信息,但很难准确地识别它们. 在AI可靠地用于检测这些掠夺性出版物之前,需要进一步开发.
科学领域:
- 图书计量和学术传播
- 人工智能在研究中的完整性
背景情况:
- 值得怀疑的期刊,包括被劫持的期刊,对学术诚信和研究界构成重大威胁.
- 识别被劫持的期刊对于研究人员来说至关重要,以避免掠夺性的出版实践.
- 人工智能 (AI) 聊天机器人为早期检测劫持期刊提供了一个潜在的途径.
研究的目的:
- 分析和基准各种AI聊天机器人在识别被劫持的医学期刊的性能.
- 评估AI聊天机器人在区分合法和被劫持的期刊标题和网站的能力.
主要方法:
- 收集了31个被劫持的期刊 (21个先前被发现,10个新发现的) 和它们的合法对应的数据集.
- 七个人工智能聊天机器人 (ChatGPT,Gemini,Copilot,DeepSeek,Qwen,Perplexity,Claude) 进行了基准测试. 这些机器人被认为是智能聊天机器人.
- 用三种类型的问题来评估聊天机器人提供信息的能力,识别被劫持的网站和验证合法的网站.
主要成果:
- 人工智能聊天机器人展示了能够提供关于被劫持期刊的一般信息的能力.
- 然而,目前的AI聊天机器人无法可靠地识别合法或被劫持的期刊标题.
- 与其他聊天机器人相比,Copilot表现更好,但仍然存在错误.
结论:
- 现有的AI聊天机器人还不是可靠的工具来检测被劫持的期刊.
- 有一种风险是,当前的AI聊天机器人可能会无意中推广被劫持的期刊.
- 人工智能的进一步进步是必要的,以提高识别学术出版威胁的准确性和可靠性.
相关概念视频
Errors occurring during blood pressure monitoring
1.6K
Blood pressure monitoring is a crucial clinical procedure in diagnosing and managing various cardiovascular conditions. Despite its significance, the accuracy of blood pressure measurements can be compromised by multiple factors, potentially leading to either falsely high or low readings. These inaccuracies are critical as they can significantly impact patient care. So, it is vital to understand these challenges deeply and adopt strategic approaches to minimize errors.
Several factors...
Several factors...
1.6K
Improving Translational Accuracy
3.7K
3.7K
Issues And Trends In Healthcare Delivery System
6.3K
The issues and trends in healthcare delivery are constantly changing. The COVID-19 pandemic is one recent issue that wreaked havoc on healthcare systems, causing a shortage of healthcare workers, high demand for medicines and supplies, and increased medical expenditure due to a lack of insurance. Other issues include rising healthcare costs and care fragmentation.
Cost Containment
Payment for healthcare services has historically promoted adoption of costly and often unnecessary or inefficient...
Cost Containment
Payment for healthcare services has historically promoted adoption of costly and often unnecessary or inefficient...
6.3K
Health Information Technology and Healthcare Information System
3.5K
Health Information Technology (HIT)
Health Information Technology, commonly called HIT, integrates advanced information systems and technology in healthcare settings. Its primary functions include:
Health Information Technology, commonly called HIT, integrates advanced information systems and technology in healthcare settings. Its primary functions include:
3.5K
Accuracy and Errors in Hypothesis Testing
637
Hypothesis testing is a fundamental statistical tool that begins with the assumption that the null hypothesis H0 is true. During this process, two types of errors can occur: Type I and Type II. A Type I error refers to the incorrect rejection of a true null hypothesis, while a Type II error involves the failure to reject a false null hypothesis.
In hypothesis testing, the probability of making a Type I error, denoted as α, is commonly set at 0.05. This significance level indicates a 5%...
In hypothesis testing, the probability of making a Type I error, denoted as α, is commonly set at 0.05. This significance level indicates a 5%...
637
Healthcare Agencies II
1.1K
There are various healthcare agencies in the United States—some of which are managed by religious institutions and others by different government branches.
Parish nursing is a growing specialty nursing profession that focuses on holistic healthcare, health promotion, and illness prevention. It blends professional nursing practice with a health ministry, focusing on health and healing within the context of a Christian community. Parish nurses serve as health educators, referral sources,...
Parish nursing is a growing specialty nursing profession that focuses on holistic healthcare, health promotion, and illness prevention. It blends professional nursing practice with a health ministry, focusing on health and healing within the context of a Christian community. Parish nurses serve as health educators, referral sources,...
1.1K

