Analyzing Online Discourse of Retrograde Cricopharyngeal Dysfunction: A Human-Validated LLM Study
Related Experiment Video
Updated: Oct 31, 2026
Minimally Invasive Murine Laryngoscopy for Close-Up Imaging of Laryngeal Motion During Breathing and Swallowing
Published on: December 1, 2023
Spencer T Johnson1,2, Nancy Jiang1
1Department of Otolaryngology-Head and Neck Surgery, Stanford University, Palo Alto, California, USA.
Objectives:
Perform a human-validated large language model (LLM) analysis of an online Reddit community dedicated to retrograde cricopharyngeal dysfunction (R-CPD), characterizing common themes, symptoms, treatment experiences, and diagnostic barriers.
Study Design:
Cross-sectional, observational analysis of patient-generated online discourse using LLM-based thematic classification validated against human adjudication.
Setting:
r/NoBurp, a public Reddit community dedicated to R-CPD.
Methods:
All posts from March 2014 to December 2025 were extracted and deduplicated. A 400-post subset was human-adjudicated to establish a reference standard for interrater comparison (precision, recall, F1 score, Cohen's kappa). All posts were analyzed with ChatGPT-5.2 for theme identification, temporal trends, symptoms frequency, and sentiment analysis. A bibliometric analysis compared annual R-CPD publication volume to annual Reddit post volume.
Results:
20,299 unique posts were analyzed. The five dominant themes were core symptom burden (56.5%), treatment experience (53.3%), healthcare navigation (26.0%), self-management (15.2%), and reflux-related symptoms (12.5%). Annual post volume rose sharply years before the major increase in R-CPD-related publications. Interrater agreement was high for botulinum toxin/procedural treatment (k = 0.70, 95% CI [0.64-0.77], F1 = 0.81), moderate for core symptom burden (k = 0.44, 95% CI [0.36-0.52], F1 = 0.77), and slight-to-fair for diagnostic journey/healthcare navigation (k = 0.22, 95% CI [0.15-0.29], F1 = 0.38).
Conclusion:
Patients experience prolonged diagnostic journeys marked by clinician unfamiliarity with R-CPD, often identifying their condition through online communities. Rising patient discourse preceded growth in the published literature, suggesting that r/NoBurp functioned not only as a support forum but also as an accelerator of R-CPD research.
