Exploring the value of ChatGPT in selecting antidiabetic agents for type 2 diabetes

We investigated the feasibility and effects of using large language models (LLMs), particularly GPTs, to support the selection of anti-diabetic medications for T2DM management based on individual-level clinical information.

This retrospective study included adults diagnosed with T2DM who visited the Endocrine Department at Yongin Severance Hospital. ChatGPT 4.0 was used with zero-shot and few-shot learning approaches to recommend treatments based on clinical data. Concordance between ChatGPT's recommendations and clinician prescriptions for monotherapy, dual therapy, and triple therapy was categorised as agree, partially agree, and disagree.

Among the 85 individuals included, the overall concordance rate was highest for monotherapy and decreased as the treatment regimen became more complex. In treatment-naive individuals, agreement rates were 69.2% (1st), 69.2% (2nd), and 84.6% (3rd) for monotherapy; 12.5% (1st), 20.8% (2nd), and 0% (3rd) for dual therapy; and 0% at all three assessment points for triple therapy. The concordance rate was lower for individuals with prior treatment history. Few-shot prompting improved agreement compared with zero-shot, particularly for monotherapy and dual therapy.

ChatGPT shows potential as a decision-support tool for selecting anti-diabetic medications, particularly for treatment-naive individuals. Few-shot learning demonstrated improvements in recommendation accuracy, especially for simpler regimens. However, accuracy was notably limited in complex regimens such as triple therapy, highlighting the need for further refinement before clinical use.