AI dialogues in cartilage repair: which guides evidence-based decisions better?

This study compares ChatGPT, DeepSeek, and Google Search in addressing cartilage repair-related questions across two domains-cartilage tissue engineering (CTE) and cartilage repair surgery (CRS)-using a dual-axis framework that integrates classification, blinded quality scoring, and readability analysis.

Google Search was queried for the top 20 frequently asked questions (FAQs) in each domain (CTE in 2023, CRS in 2024). The identical Top-10 Google-derived FAQs per domain were subsequently submitted to all three platforms-Google, ChatGPT (GPT-4 API), and DeepSeek (V3 API)-enabling a matched three-way comparison. Questions and answer sources were classified using a modified Rothwell taxonomy. Answer quality was independently evaluated by three blinded raters using the Accuracy-Safety-Hallucination (ASH) framework. Readability was assessed via the Flesch-Kincaid formula.

In the CTE domain, DeepSeek achieved the highest Accuracy (median 5.00, IQR 4.67-5.00) and significantly outperformed Google (median 4.00, Bonferroni-corrected p = 0.036), while ChatGPT (median 3.67) did not differ significantly from either platform. In the CRS domain, both ChatGPT (median 5.00) and DeepSeek (median 5.00) significantly outperformed Google (median 4.17; p = 0.024 and p = 0.045, respectively), with Safety significantly favoring both LLMs (Cochran's Q p = 0.018). ChatGPT and DeepSeek did not differ significantly in Accuracy in either domain. Readability analysis paradoxically favored Google (Grade Level 12.6-13.2 vs. 15.7-17.4 for LLMs), attributable to extreme snippet brevity inflating formulaic scores rather than genuine comprehensibility.

ChatGPT and DeepSeek outperform Google Search in accuracy and safety, yet their value lies in complementary functional roles rather than direct competition. ChatGPT's policy- and education-oriented framing and strong CRS safety profile position it as a practical tool for patient education. DeepSeek's technical depth and academically concentrated sourcing make it better suited for clinical decision support and research. Google offers the highest readability and closely mirrors patient concerns but carries measurable safety risks in surgical contexts. These findings advocate for stakeholder-specific AI tool matching rather than one-size-fits-all recommendations.