Hallucination Rates and Reference Accuracy of ChatGPT and Bard for Systematic Reviews: Comparative Analysis

Area of Science:

• Artificial Intelligence in Scientific Research
• Bibliometrics and Information Science
• Medical Literature Analysis

Background:

• Large language models (LLMs) offer potential for automating literature searches in systematic reviews.
• Concerns exist regarding LLM reliability and the persistent generation of unsupported (hallucinated) content.
• Assessing LLM performance in scientific reference generation is crucial for academic integrity.

Purpose of the Study:

• To evaluate the reference-generating capabilities of LLMs, specifically ChatGPT and Bard (now Gemini).
• To compare LLM performance against human-conducted systematic reviews in scientific writing.
• To quantify accuracy, recall, precision, and hallucination rates of LLM-generated references.

Main Methods:

• LLMs (GPT-3.5, GPT-4, Bard) were tested using inclusion criteria from systematic reviews on shoulder rotator cuff pathology.
• Performance was measured against gold-standard references from human-conducted systematic reviews.
• Key metrics included precision, recall, F1-score, and hallucination rate, with hallucinations defined by incorrect title, author, or year.

Main Results:

• Bard exhibited 0% precision and a 91.4% hallucination rate; GPT-3.5 and GPT-4 showed low precision (9.4%-13.4%) and high hallucination rates (28.6%-39.6%).
• Recall rates for GPT-3.5 and GPT-4 were also low (11.9%-13.7%), with Bard failing to retrieve relevant papers.
• LLMs demonstrated biases in study type, participant criteria, and geographical/open-access representation, alongside significant hallucination issues.

Conclusions:

• Current LLMs are not recommended as primary tools for systematic reviews due to poor accuracy and high hallucination rates.
• All references generated by LLMs require rigorous validation by human researchers.
• Improvements in LLM training and functionality are necessary before their reliable use in academic research.