ChatGPT-4 in Neurosurgery: Improving Patient Education Materials

Adequate understanding of health information has been shown to be a stronger determinant of health than several demographic factors, including age, income, or employment status. However, existing neurosurgical patient education materials (PEMs) may be too complex for the average American and may contribute to poor health literacy. Large language model chatbots may provide a rapid and low-cost means of rewriting existing PEMs at a lower reading level to improve patient understanding and overall health literacy.

Neurosurgical PEMs pertaining to stroke, laminectomy, pituitary tumors, epilepsy, and hydrocephalus published by the top 100 US hospitals were collected. For all PEMs, common measures of reading level and difficulty were generated, including Flesch Kincaid Grade Level, Flesch Reading Ease (FRE), Gunning Fog Index, Automated Readability Index, Coleman-Liau Index, and the Simple Measure of Gobbledygook Index readability score. ChatGPT-4 was then used to rewrite 25 randomly selected PEMs at or near the reading level of the average American (eighth-grade reading level). The rewritten PEMs were assessed for readability using the same measures of reading level and difficulty.

The mean FRE for PEMs on all 5 common neurosurgical conditions were significantly greater than corresponding scores for an eighth-grade reading level ( P < .001). The mean Kincaid value, Automated Readability Index, Coleman-Liau score, Gunning Fog Index, and Simple Measure of Gobbledygook Index for PEMs on each condition were all significantly greater than an eighth-grade reading level ( P < .01). The mean FRE score for rewritten PEMs on each topic were significantly lower than nonrewritten materials ( P < .01) except spinal stenosis ( P = .104) and were validated for accuracy.

Existing PEMs published by the top US hospitals for common neurosurgical conditions may be too complicated for the average American that reads at an eighth-grade level. Large language model chatbots can be used to efficiently rewrite these PEMs at a lower reading level while maintaining the accuracy of the material.