Artificial intelligence versus neurologists: A comparative study on multiple sclerosis expertise.

Area of Science:

• Neurology
• Artificial Intelligence
• Medical Diagnostics

Background:

• Multiple sclerosis (MS) is a complex autoimmune neurodegenerative disease impacting the central nervous system.
• Current MS diagnosis relies on imaging and cerebrospinal fluid analysis due to a lack of definitive biomarkers.
• Despite treatment advances, diagnostic and therapeutic challenges persist for neurologists.

Purpose of the Study:

• To evaluate the accuracy and knowledge scope of artificial intelligence (AI) platforms compared to neurologists in multiple sclerosis (MS).
• To assess AI's performance in MS diagnosis, treatment options, and management strategies.
• To benchmark AI capabilities against neurology specialists and residents.

Main Methods:

• Twenty multiple-choice questions on MS diagnosis and management were developed by neurology academics.
• The questions were administered to 37 neurology specialists and 79 neurology residents in Turkey.
• The same questions were posed to multiple AI platforms, including ChatGPT-4.0, GPT-4o, Gemini 1.5 Pro, Claude 3.5, Perplexity, and Perplexity Pro.

Main Results:

• Neurology specialists averaged 12.05 correct answers, while residents averaged 9.08.
• Neurology residents with over two years of training scored 11.96 correct answers.
• AI platforms achieved an average score of 17.0 out of 20, with Claude 3.5 performing highest.
• Specialists actively working in MS clinics significantly outperformed other neurologists, scoring 17.67.

Conclusions:

• AI demonstrates significant potential to aid in the diagnosis and treatment of multiple sclerosis.
• AI tools can complement the expertise of neurologists, particularly in areas of knowledge recall.
• Further investigation is required to fully understand AI's capabilities and limitations in managing complex MS cases.