Preferences of Chinese Dermatologists for Large Language Model Responses in Clinical Psoriasis Scenarios: A...

Large language models (LLMs) have shown considerable promise in supporting clinical decision-making. However, their adoption and evaluation in dermatology remains limited. This study aimed to explore the preferences of Chinese dermatologists regarding LLM-generated responses in clinical psoriasis scenarios and to assess how they prioritize key quality dimensions, including accuracy, traceability, and logicality.

A cross-sectional, web-based survey was conducted between December 25, 2024, and January 22, 2025, following the Checklist for Reporting Results of Internet E-Surveys guidelines. A total of 1247 valid responses were collected from practicing dermatologists across 33 of China's provincial-level administrative divisions. Participants evaluated responses to five categories of clinical questions (etiology, clinical presentation, differential diagnosis, treatment, and case study) generated by five LLMs: ChatGPT-4o, Kimi.ai, Doubao, ZuoYiGPT, and Lingyi-agent. Statistical associations between participant characteristics and model preferences were examined using chi-square tests.

ChatGPT-4o (Model 1) emerged as the most preferred model across all clinical tasks, consistently receiving the highest number of votes in case study (n = 740), clinical presentation (n = 666), differential diagnosis (n = 707), etiology (n = 602), and treatment (n = 656). Significant variation in model preference by professional title was observed only for the differential diagnosis task (χ ² = 21.13, df = 12, p = 0.0485), while no significant differences were found across hospital tiers (p > 0.05). In terms of evaluation dimensions, accuracy was most frequently rated as "very important" (n = 635). A significant association existed between hospital tier and the most valued dimension (χ ² = 27.667, df = 9, p = 0.0011), with dermatologists in primary hospitals prioritizing traceability more than their peers in higher-tier hospitals. No significant associations were found across professional titles (p = 0.127).

Chinese dermatologists suggest a strong preference for ChatGPT-4o over domestic LLMs in psoriasis-related clinical tasks. While accuracy remains the primary criterion, traceability and logicality are also critical, particularly for clinicians in lower-tier hospitals. These findings suggest that future clinical LLMs should prioritize not only content accuracy but also source transparency and structural clarity to meet the diverse needs of different clinical settings.