Search research articles

ABOUT JoVE

Overview Leadership Blog JoVE Help Center

AUTHORS

Publishing Process Editorial Board Scope & Policies Peer Review FAQ Submit

LIBRARIANS

Testimonials Subscriptions Access Resources Library Advisory Board FAQ

RESEARCH

JoVE Journal Methods Collections JoVE Encyclopedia of Experiments Archive

EDUCATION

JoVE Core JoVE Business JoVE Science Education JoVE Lab Manual Faculty Resource Center Faculty Site

Terms & Conditions of Use

Related Concept Videos

Non-equilibrium in the Cell

Non-equilibrium in the Cell

An important concept in studying metabolism and energy is that of chemical equilibrium. Most chemical reactions are reversible. They can proceed in both directions, releasing energy into their environment in one direction, and absorbing it from the environment in the other direction. The same is true for the chemical reactions involved in cell metabolism, such as the breaking down and building up of proteins into and from individual amino acids, respectively. Reactants within a closed system...

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by

Same author

Perinatal and Parental Predictors of Wheezing in the First Year of Life: A Saudi Arabian Birth Cohort Study.

Healthcare (Basel, Switzerland)·2026

Same author

Adapting Evidence-Based Practice Guidelines for Sedation, Analgesia, Withdrawal, and Delirium Assessment and Management in Critically Ill Children.

Critical care research and practice·2026

Same author

Validity and Reliability of the Arabic Voice Activity and Participation Profile (A-VAPP) Questionnaire.

Saudi medical journal·2026

Same author

Management of Dysphagia in Patients with Tracheostomy: Clinical Position Statement of the UEP Swallowing Committee.

Dysphagia·2026

Same author

Valganciclovir Dosing Strategies for Cytomegalovirus Prophylaxis in Pediatric Solid Organ Transplant Recipients: A Comparative Single-Center Study.

Viruses·2026

Same author

Travel-Related Challenges of Chikungunya Virus and Vaccination Options.

Journal of epidemiology and global health·2026

Search research articles

Related Experiment Video

Updated: May 6, 2026

Assessment and Communication for People with Disorders of Consciousness

Assessment and Communication for People with Disorders of Consciousness

Published on: August 1, 2017

Reference Hallucination Score for Medical Artificial Intelligence Chatbots: Development and Usability Study.

Fadi Aljamaan¹, Mohamad-Hani Temsah¹, Ibraheem Altamimi¹

¹College of Medicine, King Saud University, Riyadh, Saudi Arabia.

JMIR Medical Informatics

|July 31, 2024

Summary

A new reference hallucination score (RHS) evaluates AI chatbot citations in medical research. Elicit and SciSpace showed low hallucination, while ChatGPT and Bing exhibited critical levels, highlighting the need for verification tools.

Keywords:

Bing ChatGPT Elicit Perplexity SciSpace artificial intelligence (AI) chatbots bibliographic verification reference hallucination

More Related Videos

Evaluating Usability Aspects of a Mixed Reality Solution for Immersive Analytics in Industry 4.0 Scenarios

Evaluating Usability Aspects of a Mixed Reality Solution for Immersive Analytics in Industry 4.0 Scenarios

Published on: October 6, 2020

The Immersive Cleveland Clinic Virtual Reality Shopping Platform for the Assessment of Instrumental Activities of Daily Living

The Immersive Cleveland Clinic Virtual Reality Shopping Platform for the Assessment of Instrumental Activities of Daily Living

Published on: July 28, 2022

Related Experiment Videos

Last Updated: May 6, 2026

Assessment and Communication for People with Disorders of Consciousness

Assessment and Communication for People with Disorders of Consciousness

Published on: August 1, 2017

Evaluating Usability Aspects of a Mixed Reality Solution for Immersive Analytics in Industry 4.0 Scenarios

Evaluating Usability Aspects of a Mixed Reality Solution for Immersive Analytics in Industry 4.0 Scenarios

Published on: October 6, 2020

The Immersive Cleveland Clinic Virtual Reality Shopping Platform for the Assessment of Instrumental Activities of Daily Living

The Immersive Cleveland Clinic Virtual Reality Shopping Platform for the Assessment of Instrumental Activities of Daily Living

Published on: July 28, 2022

Area of Science:

Medical research
Artificial intelligence
Scientific writing

Background:

AI chatbots are increasingly used by healthcare practitioners.
AI chatbot outputs, including references, may contain hallucinations.
These inaccuracies raise concerns about AI reliability in medical contexts.

Purpose of the Study:

To introduce a novel reference hallucination score (RHS) for assessing the authenticity of AI chatbot-generated citations.
To quantify and compare citation hallucination across different AI chatbots.

Main Methods:

Six AI chatbots were tested with 10 medical prompts each, requesting 10 references per prompt.
The RHS was developed, considering bibliographic details and relevance to prompt keywords.
RHS was calculated for individual references, prompts, and prompt types (basic vs. complex).

Main Results:

Bard did not generate references. Elicit and SciSpace had the lowest RHS (1), while ChatGPT 3.5 and Bing had the highest (11).
Reference relevancy to prompt keywords was the most common hallucination (61.6%).
AI chatbots produced significantly higher hallucination scores for complex or scenario-based prompts.

Conclusions:

The study highlights significant variations in AI chatbot citation authenticity, necessitating robust evaluation tools.
Elicit and SciSpace demonstrated minimal hallucination, whereas ChatGPT and Bing showed critical levels.
The proposed RHS can aid in improving the reliability and trustworthiness of AI in medical research.