Related Experiment Video
Updated: Jun 16, 2026
Digital Home-Monitoring of Patients after Kidney Transplantation: The MACCS Platform
Published on: April 12, 2021
Artificial intelligence chatbots in response to patient's common inquiries about chordoma: A cross-sectional study
Shaohui He1, Dongjie Jiang1, Runyi Jiang1
1Spinal Tumor Center, Department of Orthopaedic Oncology, Changzheng Hospital, Naval Medical University (Second Military Medical University), Shanghai, 200003, China.
Introduction:
Artificial intelligence (AI) chatbots has been well studied in many common diseases. However, little was reported for chordoma, which is a rare disease with high rates of recurrence, disability, and mortality.
Research Question:
This study aims to assess the performance of oncologists and state-of-the-art AI chatbots in response to frequently-asked questions (FAQs) of chordoma from real world. The response performance was mainly evaluated by overall quality, empathy, and readability.
Material And Methods:
Sixty chordoma-related FAQs, collected from social media, were addressed by various chatbots and oncologists, and the best-performing chatbot-generated text was further edited and assessed again. Rated scores were ordered for quality and empathy in a blind way. The readability was measured objectively by calculating Flesch-Kincaid Grade Level (FKGL), Automated Readability Index (ARI), and Gunning-Fog Index (GFI).
Results:
AI chatbots were universally superior to oncologists in response quality (3.86 ± 0.14 vs. 3.12 ± 0.25, p < 0.001) and empathy (3.28 ± 0.41 vs. 2.95 ± 0.48, p < 0.001). DeepSeek-R1 achieved highest rated score in response quality (4.20 ± 0.22), while Claude 3.5 Sonnet was considered as the best chatbots by comprehensive assessments. The chatbot drafted responses were easier to understand from patient's perspective (p < 0.001). Improved response quality (4.09 ± 0.12, p < 0.001), empathy (4.00 ± 0.39, p < 0.001), and readability (FKGL: 11.30 ± 2.42, p < 0.001) were obtained after editing the Claude-3.5-generated responses by oncologists.
Discussion And Conclusions:
AI chatbots reached favorable quality and empathetic performance in response to chordoma-related FAQs, and generated equivalent readability compared to oncologists. With generative chatbot's assistance, oncologists may response more comprehensively and efficiently in addressing chordoma patient's common inquiries.
Insights
Artificial intelligence (AI) chatbots significantly outperformed oncologists in responding to chordoma patient questions, demonstrating superior quality and empathy. AI assistance can help oncologists provide more comprehensive and efficient patient care for this rare cancer.
Area of Science:
- Oncology
- Artificial Intelligence
- Medical Informatics
Background:
- Chordoma is a rare cancer with high recurrence, disability, and mortality rates.
- Limited research exists on AI chatbot performance for rare diseases like chordoma.
Purpose of the Study:
- To evaluate AI chatbots and oncologists' responses to chordoma-related frequently asked questions (FAQs).
- To compare response quality, empathy, and readability between AI and human experts.
Main Methods:
- Sixty chordoma FAQs from social media were used.
- AI chatbots and oncologists responded to FAQs; best AI responses were edited by oncologists.
- Responses were evaluated for quality and empathy (blind rating) and readability (FKGL, ARI, GFI).
Main Results:
- AI chatbots showed superior response quality (p<0.001) and empathy (p<0.001) compared to oncologists.
- Claude 3.5 Sonnet was the top-performing chatbot; DeepSeek-R1 led in quality.
- Edited AI responses improved quality, empathy, and readability.
Conclusions:
- AI chatbots offer high quality and empathetic responses to chordoma FAQs, with comparable readability to oncologists.
- AI chatbots can assist oncologists in addressing common patient inquiries efficiently and comprehensively.