Trust, truth and transparency: analysing the references underpinning AI-generated surgical information

Area of Science:

• Medical Informatics
• Artificial Intelligence in Healthcare
• Scientific Communication

Background:

• Growing use of AI chatbots for surgical information dissemination.
• Concerns regarding accuracy, hallucinations, and source reliability of AI-generated medical content.
• Need to assess AI-generated information sources for transparency and evidence-based healthcare integration.

Purpose of the Study:

• To evaluate the sources of information used by AI chatbots for medical content generation.
• To assess the quantity, authenticity, quality, and accessibility of references provided by AI systems.
• To understand the impact of explicit prompting on reference generation in AI chatbots.

Main Methods:

• Nine AI chatbots were queried with standardized general surgery prompts.
• Outputs were analyzed for reference quantity, authenticity, quality, source category, accessibility, geographic origin, and attribution.
• 1,249 references were extracted and assessed across 108 chatbot outputs.

Main Results:

• Reference provision varied significantly among chatbots, with some requiring explicit prompting.
• Hallucination rates ranged from 0% to 34%.
• Perplexity Research had the highest mean quality score (4.08), ChatGPT-5 the lowest (2.39). Google Gemini offered 100% open-access citations.

Conclusions:

• AI chatbots demonstrate heterogeneous reference integrity, posing risks of hallucinations and bias.
• Prompt engineering, model refinement, and continuous evaluation are essential for safe AI use in surgery.
• Caution is advised for AI dissemination of surgical information to ensure safety and equity.