Evaluation over Generalist Large Language Models and Specialised Models for Clinical Risk Prediction

Large language models (LLMs) show promise in clinical applications but face accuracy limitations in disease risk prediction, especially for rare conditions. We evaluated ChatGPT and DeepSeek against task-specific models for cholangiocarcinoma (rare) and myocardial infarction (common) prediction. Standalone LLMs underperformed specialized models, whereas collaborative prediction improved performance. Our findings suggest hybrid architectures leveraging complementary strengths of both approaches represent a promising direction for clinical AI deployment.