Related Experiment Video
Updated: Mar 1, 2026

Evidence-based Knowledge Synthesis and Hypothesis Validation: Navigating Biomedical Knowledge Bases via Explainable AI and Agentic Systems
Published on: June 13, 2025
Benchmarking AI chatbots: assessing their accuracy in identifying hijacked medical journals
Mihály Hegedűs1,2, Mehdi Dadkhah3, Lóránt Dénes Dávid3,4,5,6
1Department of Finance and Accounting, Tomori Pál College, Budapest, Hungary.
Objectives:
The challenges posed by questionable journals to academia are very real, and being able to detect hijacked journals would be valuable to the research community. Using an artificial intelligence (AI) chatbot may be a promising approach to early detection. The purpose of this research is to analyze and benchmark the performance of different AI chatbots in identifying hijacked medical journals.
Methods:
This study utilized a dataset comprising 21 previously identified hijacked journals and 10 newly detected hijacked journals, alongside their respective legitimate versions. ChatGPT, Gemini, Copilot, DeepSeek, Qwen, Perplexity, and Claude were selected for benchmarking. Three question types were developed to assess AI chatbots' performance in providing information about hijacked journals, identifying hijacked websites, and verifying legitimate ones.
Results:
The results show that current AI chatbots can provide general information about hijacked journals, but cannot reliably identify either real or hijacked journal titles. While Copilot performed better than others, it was not error-free.
Conclusions:
Current AI chatbots are not yet reliable for detecting hijacked journals and may inadvertently promote them.
Related Concept Videos
Errors occurring during blood pressure monitoring
Several factors...
Improving Translational Accuracy
Issues And Trends In Healthcare Delivery System
Cost Containment
Payment for healthcare services has historically promoted adoption of costly and often unnecessary or inefficient...
Health Information Technology and Healthcare Information System
Health Information Technology, commonly called HIT, integrates advanced information systems and technology in healthcare settings. Its primary functions include:
Accuracy and Errors in Hypothesis Testing
In hypothesis testing, the probability of making a Type I error, denoted as α, is commonly set at 0.05. This significance level indicates a 5%...
Healthcare Agencies II
Parish nursing is a growing specialty nursing profession that focuses on holistic healthcare, health promotion, and illness prevention. It blends professional nursing practice with a health ministry, focusing on health and healing within the context of a Christian community. Parish nurses serve as health educators, referral sources,...
