Related Experiment Video
Updated: Jun 16, 2026

An Affordable HIV-1 Drug Resistance Monitoring Method for Resource Limited Settings
Published on: March 30, 2014
Performance comparison of large language models for medication counseling in people living with HIV
Can Huang1, Yanfang Sun1, Meng Chen1
1Beijing Youan Hospital, Capital Medical University, Beijing, China.
Objective:
This study aimed to evaluate the comprehensive performance of five large language models (LLMs), namely ChatGPT, DeepSeek, Doubao, Kimi, and Qwen, in addressing medication consultation inquiries for people living with HIV (PLWH) in a Chinese-language context, thereby providing evidence for their clinical application and further model optimization.
Methods:
A total of 55 real-world medication consultation questions covering mainstream antiretroviral drugs for PLWH were screened and classified from Beijing Youan Hospital, Capital Medical University, a specialized infectious disease hospital in China. Five LLMs were queried within a fixed period, and expert evaluations were conducted across five dimensions: accuracy, relevance, completeness, clarity, and reliability.
Results:
The comprehensive scores ranked from highest to lowest were DeepSeek (4.47), Qwen (4.33), Kimi (4.24), Doubao (4.13), and ChatGPT (3.41), with highly significant differences were observed among all models (H=182.14, p < 0.001). Regarding dimensional scores, the ranking was clarity (4.53) > reliability (4.33) > completeness (4.13) > accuracy (3.95) > relevance (3.62). ChatGPT exhibited statistically significant differences compared with all other models (p < 0.001); no significant differences were found between DeepSeek and Qwen or between Doubao and Kimi (p > 0.05).
Conclusion:
Significant differences were observed in the capacity of the five LLMs to address medication consultations for PLWH within the Chinese-language context. DeepSeek and Qwen achieved optimal overall performance, Doubao excelled in clarity, whereas ChatGPT yielded the poorest results. All models demonstrated significant limitations when handling complex pharmaceutical inquiries and cannot fully replace professional clinical pharmacists. Further optimization focusing on high-quality medical domain dataset training and algorithm refinement is therefore warranted.
Related Concept Videos
Pharmacokinetic Models: Comparison and Selection Criterion
Physiological models take a detailed approach by considering specific molecular processes. They can predict drug distribution, metabolism, and elimination changes, providing a comprehensive understanding of how drugs interact with the body.
Dosage Regimens: Designs and Approaches
Drug Therapy
Antianxiety Medications
Drug Dosage Regimen: Overview
Typically, the starting dose and dosing interval are guided by the manufacturer's recommendations based on clinical trials conducted during and after drug...
Therapeutic Drug Monitoring: Affecting Factors
Retrovirus Life Cycles
