Comparison of artificial intelligence and multidisciplinary team assessment for evaluating biologic decisions in...

The management of severe asthma with biologics has become more complex involving multidisciplinary team meetings. Artificial intelligence has been proposed as an adjunct to aid healthcare professionals in other areas of the medical field.

Our aim is to determine if there is a role for artificial intelligence, as an adjunct to the severe asthma multidisciplinary team meeting, to offer improved consistency and efficiency in biologic prescribing.

This is a retrospective, unblinded single-centre, multisite cohort study including patients with uncontrolled severe GINA defined stage 5 asthma for biologic consideration who were biologic naïve. They were discussed at joint multidisciplinary team meetings between secondary care referral centres in the health board NHS Tayside, Scotland between January 2024 and August 2025. The multidisciplinary team biologic decisions were compared to five large language model recommendations (ChatGPT, Gemini, CoPilot, PerplexityAI and DeepSeek) using Cohen's kappa with the multidisciplinary team meeting as the reference.

Among 114 patients, the mean (SD) age was 54·5 (15·0) years, asthma control questionnaire score 2·98 (1·1), annualised exacerbation rate 5·21 (2·1). Multidisciplinary team decisions and artificial intelligence recommendations showed poor agreement (κ = 0·143-0·213). Agreement between artificial intelligence models was weak to moderate (κ = 0·348-0·586) with frequent decision making errors identified with all artificial intelligence models.

At the time of analysis, artificial intelligence models show poor concordance highlighting their unsuitability as either a standalone or adjunctive decision maker for biologic therapy. Multidisciplinary team meetings remain the reference standard.