Clinical performance and practical limitations of large language models in respiratory medicine

This Journal club discusses three recent studies evaluating large language models (LLMs) in respiratory medicine and allergy. In COPD, ChatGPT-3.5 was rated higher than Bing Chat by respiratory specialists for accuracy, completeness, clarity and safety, although clinically important omissions remained. In asthma, 13 out of 14 models exceeded 85% accuracy on multiple-choice questions and five exceeded 95%, but performance was lower and more variable in objective structured clinical examinations. In allergen immunotherapy, ChatGPT-4 achieved fair overall DISCERN quality, but showed insufficient Journal of the American Medical Association (JAMA) benchmark reliability and produced responses that were often difficult to read. Together, these studies show that LLMs perform best on structured and familiar knowledge tasks, while weaknesses persist in completeness, transparency, readability and clinically nuanced application. Current evidence supports supervised educational use in respiratory medicine and allergy, but not unsupervised patient counselling or autonomous clinical deployment.