Reference Hallucination Score for Medical Artificial Intelligence Chatbots: Development and Usability Study

Area of Science:

• Medical research
• Artificial intelligence
• Scientific writing

Background:

• AI chatbots are increasingly used by healthcare practitioners.
• AI chatbot outputs, including references, may contain hallucinations.
• These inaccuracies raise concerns about AI reliability in medical contexts.

Purpose of the Study:

• To introduce a novel reference hallucination score (RHS) for assessing the authenticity of AI chatbot-generated citations.
• To quantify and compare citation hallucination across different AI chatbots.

Main Methods:

• Six AI chatbots were tested with 10 medical prompts each, requesting 10 references per prompt.
• The RHS was developed, considering bibliographic details and relevance to prompt keywords.
• RHS was calculated for individual references, prompts, and prompt types (basic vs. complex).

Main Results:

• Bard did not generate references. Elicit and SciSpace had the lowest RHS (1), while ChatGPT 3.5 and Bing had the highest (11).
• Reference relevancy to prompt keywords was the most common hallucination (61.6%).
• AI chatbots produced significantly higher hallucination scores for complex or scenario-based prompts.

Conclusions:

• The study highlights significant variations in AI chatbot citation authenticity, necessitating robust evaluation tools.
• Elicit and SciSpace demonstrated minimal hallucination, whereas ChatGPT and Bing showed critical levels.
• The proposed RHS can aid in improving the reliability and trustworthiness of AI in medical research.