Cross-platform evaluation of LLM-generated educational texts on cardiac myxoma: quality, readability, and...

Patients with cardiac myxoma require long-term follow-up, and the quality of patient education as well as the ability to recognize early symptoms may influence integrated postoperative cardiovascular and tumor-related management. Although large language models (LLMs) have been increasingly applied in medical education, cross-platform empirical evidence in this intersecting context remains limited. This study evaluated the performance and limitations of widely used LLMs in generating patient education texts for cardiac myxoma using a standardized, expert-curated educational question set.

We constructed a standardized dataset of 60 expert-curated educational questions spanning the disease course of cardiac myxoma and used nine widely used LLMs to generate 540 patient education texts. Text quality, readability, and actionability were assessed using the Patient Education Materials Assessment Tool for Printable Materials (PEMAT-P), Ensuring Quality Information for Patients (EQIP-36), the Global Quality Score (GQS), and seven readability formulas. In addition, a regularized Gaussian graphical model-based partial correlation network analysis and latent profile analysis were performed to identify relationships among evaluation metrics and cross-platform text phenotypes.

Texts generated across platforms showed significant heterogeneity in information quality, understandability, and objective readability, whereas clinical actionability was generally low. Word count showed the strongest positive correlation with the total EQIP-36 score and occupied a central position in the network. Reading-difficulty indices were consistently negatively correlated with PEMAT-P actionability. Latent profile analysis identified three text phenotypes: moderate-quality/low-readability, high-quality/high-actionability, and low-quality/easy-to-read. Ideally suited patient education texts accounted for only a very small proportion of all outputs.

Within this expert-curated educational question set, the current application of LLMs in patient education for cardiac myxoma is primarily limited by reading burden and insufficient behavioral guidance. Although longer outputs were generally associated with higher informational quality scores, greater syntactic complexity was associated with lower actionability of the materials. In addition, latent profile analysis suggested that only a small subset of outputs approached a more favorable quality-actionability profile.