Evaluating AI chatbots in neurological function test interpretation for brain tumor surgery

Neuropsychological assessments are essential for evaluating functional status and guiding surgical planning in patients with brain tumors. However, their complexity may hinder interpretation for patients and junior clinicians. Large language model (LLM)-based chatbots have emerged as tools providing medical information, but their ability to interpret real-world neuropsychological test results remains unevaluated. This study investigates whether LLMs can provide accurate, patient-friendly explanations of neuropsychological tests and support communication in neurosurgical care. We included 20 patients who underwent at least one of five neuropsychological tests (Seoul Neuropsychological Screening Battery, Albert Test, Line Bisection Test, Boston Naming Test, or Western Aphasia Battery) prior to brain tumor surgery. Three LLMs (ChatGPT, Copilot, and Perplexity) were prompted with standardized queries on test explanations. Responses were evaluated for readability using Flesch-Kincaid Grade Level, understandability using a modified Patient Education Materials Assessment Tool, and explanatory accuracy using an expert-rated 4-point scale. Readability scores ranged from 9.6 (Copilot) to 11.0 (Perplexity). Understandability scores were highest for ChatGPT (83.2%), followed by Perplexity (81.3%) and Copilot (66.4%). ChatGPT performed best in test purpose and methodology; Perplexity scored highest in result interpretation and average accuracy score across domains. Patient rated Perplexity highly in understandability (4.0/4.0) and usefulness (3.8/4.0). LLMs generated accurate, understandable explanations of neuropsychological tests. These tools may support multidisciplinary care and patient communication in brain tumor surgery.