Evaluating ChatGPT for neurocognitive disorder diagnosis: a multicenter study.

Area of Science:

• Neuroscience
• Artificial Intelligence
• Medical Diagnostics

Background:

• Neurocognitive disorders pose a significant public health challenge.
• Accurate and accessible diagnostic tools are essential for timely intervention.
• Artificial intelligence (AI) shows promise in augmenting clinical decision-making.

Purpose of the Study:

• To evaluate the diagnostic accuracy and reliability of ChatGPT 4 Omni for neurocognitive disorders.
• To compare the performance of ChatGPT 4 Omni against previous ChatGPT versions.
• To assess AI's potential in neuropsychological assessment.

Main Methods:

• Two studies were conducted: Study 1 used a few-shot prompt approach for diagnostic agreement, and Study 2 used a zero-shot prompt approach for diagnostic performance comparison.
• Data from the National Alzheimer's Coordinating Center (NACC) Uniform Data Set 3 was utilized.
• Participants included older adults diagnosed with no cognitive impairment, mild cognitive impairment (MCI), or dementia.

Main Results:

• ChatGPT 4 Omni demonstrated fair diagnostic agreement with clinicians in Study 1 (κ = .33), with MoCA and memory recall tests being notable predictors.
• High internal reliability (α = .96) was observed for ChatGPT 4 Omni.
• No significant diagnostic agreement was found between ChatGPT versions and clinicians in Study 2.

Conclusions:

• ChatGPT 4 Omni shows potential but is currently insufficient for independent clinical diagnosis of neurocognitive disorders.
• Continued AI model refinement and comprehensive training are necessary for effective neuropsychological assessment.
• AI integration in clinical practice may enhance diagnostic efficiency and access to services in the future.