Evaluation of AI-Based Chatbots in Liver Cancer Information Dissemination: A Comparative Analysis of GPT, DeepSeek,...

This study aimed to evaluate AI-based chatbots (GPT, DeepSeek, Copilot, Gemini) in disseminating information on liver cancer, emphasizing content quality, adherence to established guidelines, and ease of comprehension.

Between January and February 2025, four chatbots were examined using publicly accessible free versions lacking independent reasoning capabilities. Three frequently searched Google Trends questions ("What is liver cancer awareness?," "What are the symptoms of liver cancer?", and "Is liver cancer treatable?") were posed. Their responses were assessed via the DISCERN instrument, Coleman-Liau Index, Patient Education Materials Assessment Tool for Print, and alignment with American Association for the Study of Liver Diseases, National Comprehensive Cancer Network, and European Society for Medical Oncology recommendations. Statistical analysis was performed using SPSS 22.

All chatbots largely provided relevant and impartial information. GPT and DeepSeek scored lower on specifying information sources and update timelines, whereas Copilot omitted local therapies (e.g., radiofrequency ablation, transarterial chemoembolization, transarterial radioembolization), resulting in reduced scientific accuracy. Gemini and Copilot performed better in "understandability," while GPT and DeepSeek excelled in "actionability." Although GPT demonstrated consistency across multiple treatment options, it did not explicitly reference international guidelines. Study limitations included language constraints, variations in chatbot updates, and reliance on a single inquiry round.

AI chatbots show potential as initial informational tools for liver cancer but cannot replace professional medical consultation. In complex diseases requiring multidisciplinary management, frequent guideline-based updates, expert validation, and diverse data sources are critical to enhancing clinical relevance and patient outcomes.