Artificial intelligence chatbots as information sources for postherpetic neuralgia: An evaluation of reliability and...

Artificial intelligence (AI) chatbots are increasingly used for online health information seeking and have become a common source of information on postherpetic neuralgia (PHN). The quality and readability of the information they provide have not been systematically evaluated.

To assess the quality, transparency, and readability of postherpetic neuralgia information generated by mainstream artificial intelligence chatbots and to examine whether this content meets the recommended sixth-grade reading level for patient education materials.

Using the MeSH term Neuralgia, Postherpetic, we identified 11 highly relevant postherpetic neuralgia-related search queries worldwide from 2020 to 2025 through Google Trends. These queries were entered verbatim into ChatGPT-4o, Gemini-1.5, Perplexity Pro, and Copilot using a standardised input procedure. Responses were assessed with DISCERN, EQIP, JAMA, and GQS for information quality, completeness, transparency, and overall educational value. Readability was evaluated with ARI, FRES, GFI, FKGL, CL, and SMOG. The Kruskal-Wallis test was used for between-model comparisons, and the Wilcoxon signed-rank test was used to compare readability indices against sixth-grade reading benchmarks.

DISCERN, EQIP, and GQS scores differed significantly across chatbots in DISCERN (P < 0.001), EQIP (P < 0.001), and GQS (P < 0.001) scores among the four chatbots; JAMA scores showed no significant difference (P = 0.668). No chatbot reached an excellent overall rating, although Perplexity Pro and Copilot performed better than ChatGPT-4o and Gemini-1.5 on most quality-related measures. For readability, none of the chatbots met the sixth-grade reading standard on any metric: all FRES scores were <80, and all other indices (ARI, GFI, CL, FKGL, SMOG) exceeded the grade-6 thresholds. Readability did not differ significantly among chatbots (all P > 0.2).

Mainstream artificial intelligence chatbots can provide postherpetic neuralgia information with moderate quality, but the readability of this content remains consistently above recommended patient education levels. The findings should be interpreted within the scope of the selected chatbots, default operational settings, search queries, and the time-sensitive nature of chatbot outputs. Further improvement in source reporting and plain-language presentation remains warranted.