Large Language Models in Neurology Treatment Decision-Making: a Scoping Review

This scoping review evaluates the expanding role of large language models (LLMs) in neurology, an area drawing growing interest of researchers and clinicians alike. A substantial existing body of literature supports the efficacy of LLMs for diagnostic applications. However, clinicians' emerging point of interest now lies in understanding the applications of LLMs in guiding treatment decisions. Our study therefore aims to synthesize and evaluate existing neurological studies focused on LLMs in treatment decision-making. A comprehensive search was conducted in the electronic databases OVID/Medline, Web of Science, and the Cochrane Library through September 18th, 2024. Inclusion criteria included original studies published within the last five years focused on evaluating the efficacy of LLMs in treatment decision-making in neurology. The protocol was registered on the Open Science Framework ( https://doi.org/10.17605/OSF.IO/Y6N3E ). Four studies were identified. ChatGPT was the LLM utilized in each article, though varying in model versions. Each study demonstrated positive outcomes across varying metrics, with models generally aligning with clinician decisions. However, the lack of observed studies and variability of neurological topics limit the generalizability of these AI tools. This scoping review analyzes the existing body of evidence on LLMs in treatment decision-making in neurology. While current studies suggest potential to support clinical care, there is insufficient evidence at this stage to claim outcome improvement. Findings are not yet generalizable across neurological practice, as existing promise appears limited to narrow use cases. Prospective validation across subspecialties is needed to support broader clinical application.