Exploration of the assessment of clinical decision-making capabilities in Clinical Oncology based on generative large...

With advancements in large language model (LLM) technology, generative artificial intelligence (AI) has shown transformative potential in healthcare, particularly in optimizing clinical workflows through data integration and semantic reasoning for clinical decision support (CDS). However, existing AI models primarily rely on logical deductions from standardized guidelines, and their effectiveness in complex, high-risk clinical scenarios remains to be further validated. This study evaluates the CDS efficacy of DeepSeek R1 and ChatGPT-4 o1 in real-world oncology diagnosis and treatment settings, assessing the accuracy and adaptability of their recommendations through a multidimensional framework.

A time-sequenced, chain-structured clinical question set was developed based on real oncology cases. Responses from DeepSeek R1 and ChatGPT-4 o1 were independently generated and blindly evaluated for accuracy and feasibility by four oncology experts. Statistical analysis and visualization were performed using Prism GraphPad 10.0.

Both DeepSeek R1 and ChatGPT-4 o1 demonstrated overall competent performance in oncology CDS, with no significant differences compared to human clinicians. Evaluations using a clinical decision quality assessment scale indicated robust performance for both models. Subgroup analysis revealed that DeepSeek R1 outperformed in medical humanistic care, while ChatGPT-4 o1 excelled in readability. No statistical differences were observed between the two models in knowledge accuracy, test rationality, or medication standardization.

Generative AI models such as DeepSeek R1 and ChatGPT-4 o1 exhibit comprehensive capabilities in oncology CDS comparable to clinicians, suggesting potential clinical utility. However, AI reliability in complex cases requires improvement and cannot yet replace physicians' expertise. Future research should prioritize multimodal knowledge integration and ethical oversight to enhance AI's role in optimizing diagnostic efficiency and humanistic care quality.