Readability Assessment and Comparison of Large Language Model-Generated Summaries of Trial Descriptions on...

This study evaluated the readability of ClinicalTrials.gov trial information using traditional readability measures (TRMs) and compared it to summaries generated by large language models (LLMs), specifically ChatGPT and a fine-tuned BART-Large-CNN (FBLC). The study involved: 1) assessing required reading levels (RRL) with TRMs, 2) generating sample LLM-based summaries, and 3) evaluating summary quality based on scores provided by two independent reviewers. The results show that the original ClinicalTrials.gov trial descriptions were scored above the recommended readability level. In contrast, ChatGPT-generated summaries had significantly lower RRLs and higher quality scores. We conclude that ChatGPT shows great promise of creating readable, high-quality summaries. Future research is warranted to assess whether LLMs could be a viable solution to improve the readability of ClinicalTrials.gov to facilitate comprehension by laypersons.