Feasibility and impact of a large language model pipeline for surgical trial abstracts

Abstracts of surgical randomised controlled trials (RCTs) are widely read but often omit key methodological details despite CONSORT guidance. We evaluated whether a tightly constrained large language model (LLM) pipeline could improve completeness and readability at scale. We conducted a three-phase in silico study of consecutive surgical RCTs indexed in PubMed (2005-2025) with open-access full texts in PubMed Central. A 14-item CONSORT-derived rubric (maximum 25 points) was developed and validated against expert scoring, demonstrating good agreement (concordance correlation coefficient 0.71, 95% CI 0.44-0.86) and high reproducibility (intraclass correlation coefficient 0.91, 95% CI 0.80-0.96). An automated pipeline using GPT-4o (OpenAI) generated rewritten abstracts from full texts under strict non-fabrication constraints. Among 651 RCTs, original abstracts showed low completeness (mean 9.06/25, 95% CI 8.58-9.53). Rewriting significantly improved completeness (mean increase 7.40 for 250-word and 8.06 for 300-word versions; both p < 0.0001), with gains across all CONSORT domains, particularly randomisation, harms, and trial registration. Readability improved slightly and correlated with completeness. A constrained LLM pipeline can substantially enhance the completeness of surgical RCT abstracts at scale, with potential applications in authoring, peer review, and editorial workflows, provided appropriate human oversight.