Comparative informative capacity of artificial intelligence (AI)-powered chatbots in colorectal cancer: ChatGPT-4
Nurhilal Kızıltoprak1, Ömer Faruk Özkan1, Fevzi Cengiz2
1Sultan Abdülhamid Han Training and Research Hospital, Istanbul, Turkey.
Introduction:
Artificial intelligence (AI)-powered chatbots, such as ChatGPT-4 and DeepSeek, are increasingly utilized in providing medical information. However, their accuracy, comprehensiveness, and reliability, particularly in specialized fields such as colorectal cancer, remain under-evaluated. This study aimed to compare the performance of ChatGPT-4 and DeepSeek in responding to both community- and expert-oriented questions related to colorectal cancer.
Materials And Methods:
A total of 30 questions were formulated based on clinical experience, including 15 community-focused and 15 expert-oriented questions. On February 13, 2025, ChatGPT-4 (OpenAI, version 4.0) and DeepSeek-R1 (initial January 2025 release) were queried simultaneously in a single session. Responses were independently evaluated by four colorectal surgery experts for appropriateness (0-100), comprehensiveness (0-100), and reference provision (yes/no). Statistical analyses included Mann-Whitney U and chi-square tests, with significance set at p < 0.05.
Results:
ChatGPT-4 and DeepSeek demonstrated comparable appropriateness scores (94.0 vs. 92.25, p > 0.05). In community-oriented questions, ChatGPT-4 showed significantly higher comprehensiveness (median 95.0, interquartile range (IQR) 92-98 vs. 90.0, interquartile range 85-94; p = 0.044). Neither chatbot provided scientific references. Inter-rater agreement ranged from good to moderate, with slightly higher consistency observed for DeepSeek (appropriateness ICC 0.83 vs. 0.81).
Discussion:
Both chatbots exhibited distinct strengths and limitations. ChatGPT-4 demonstrated superior comprehensiveness in community-oriented responses, whereas DeepSeek provided slightly more consistent evaluations. The absence of scientific references represents a major limitation, restricting clinical applicability and reliability. Enhancing reference support and response consistency is essential before AI-powered chatbots can be safely integrated into colorectal cancer-related clinical decision-making.
More Related Videos
Related Concept Videos
Mouse Models of Cancer Study
The development of transgenic, knockout, and knock-in mice has led to an exponential increase in their use as model organisms in research,...
Adaptive Mechanisms in Cancer Cells
Some of the advantages that cancer cells have on normal cells include - enhanced ability to divide without terminally differentiating, induce new blood vessel formation,...
Tumor Immunotherapy
Cancer Prevention
Some...
Cancer Therapies
However, cancer treatments can pose several challenges, as therapies used to kill cancer cells are generally also toxic to normal cells. Moreover, cancer cells mutate rapidly and can develop resistance to chemical agents or radiation therapy. Besides, all types of cancer cells may not respond to the same therapy. Some cancer cells respond to one...
Cancer-Critical Genes II: Tumor Suppressor Genes
When the function of certain critical genes, especially those involved in cell cycle regulation and cell growth signaling cascades, gets disrupted, it upsets the cell cycle progression. Such cells with unchecked cell cycles start proliferating uncontrollably and eventually develop into tumors.
Such genes that act...


