Accuracy of Large Language Models in Extracting Literature Features for Cardiovascular Disease Exercise...

Accurate extraction of clinical and methodological features from published literature is essential for supporting evidence-based cardiovascular rehabilitation. With the rapid expansion of research outputs, manual data extraction is time-consuming and prone to errors. Large language models (LLMs) may help automate evidence synthesis tasks, but their performance in document extraction for cardiovascular rehabilitation nursing research remains unclear. We aimed to evaluate the ability of LLMs to extract key literature features in the field of cardiovascular exercise rehabilitation nursing.

A diagnostic accuracy study was conducted using a reference standard established by two independent experts in cardiovascular rehabilitation. A structured extraction framework covering study characteristics, participant profiles, intervention components, and outcome domains was applied. LLM outputs were compared with the reference standard, and sensitivity, specificity, and area under the curve (AUC) were calculated.

LLMs demonstrated high sensitivity (0.96-0.98) in identifying essential study features, indicating a low risk of missed information. Specificity ranged from moderate to high (0.70-0.77) and improved notably after incorporating nursing-specific chain-of-thought prompting. Overall diagnostic performance was excellent, with a post-hoc analysis AUC reaching 0.98. Error patterns were primarily related to ambiguous intervention descriptions and multi-component rehabilitation programs.

LLMs show strong potential as an efficient and accurate tools for automating literature feature extraction in cardiovascular exercise-based rehabilitation nursing. Their use may enhance the speed and reliability of evidence synthesis, and support clinical decision-making. Further optimisation of prompting strategies and domain-specific tuning is warranted before widespread clinical adoption.