Effect of Emotional Prompt on the Quality of ChatGPT Based Discharge Instructions After Laparoscopic Cholecystectomy...

Nurse-led education is a crucial form of discharge communication for ensuring safe postoperative in-home patient recovery. However, discharge instructions are often incomplete or inadequately tailored to patients' needs. Generative large language models (LLMs), such as ChatGPT, may support postoperative patient education.

To evaluate the completeness, accuracy, and quality of post-cholecystectomy discharge instructions, generated in Polish by ChatGPT-4 Omni, comparing a primary prompt (PP) with an emotional prompt (EP), in which motivational cues were added to the PP structure.

In this evaluation study, ChatGPT 4.0 Omni generated 60 discharge instruction 30 outputs per prompt type. Two blinded experts assessed content completeness against a 43-item benchmark derived from Enhanced Recovery After Surgery (ERAS) protocols and professional guidelines, and accuracy defined as absence of clinically incorrect, unsafe, or guideline inconsistent statements. Inter rater agreement was quantified with Cohen kappa and between prompt type differences in domain level completeness were examined with the Brunner Munzel test. Communication quality was evaluated using directed qualitative content analysis and a dictionary-based text mining approach that counted affirmatives, modal verbs, and empathetic expressions and combined them into a Composite Relationality Index (CRI) for each output.

Inter expert agreement for benchmark ratings was very high mean kappa 0.859 (95% confidence interval 0.841 to 0.878) with an overall agreement rate of 93.14%. Core domains such as analgesia, wound care, low fat diet, vital sign monitoring, and emergency procedures were present in all outputs 100%. Several elements were underrepresented, including full antibiotic prescription details 25.0%, gradual fibre reintroduction 41.7%, specification of an adaptation period 30.0%, monitoring of gastrointestinal or other red flag symptoms 33.3%, and individual dietary product groups each below 25%. No statistically significant differences in completeness were detected between primary and emotional prompts for any of the 43 features, and expert review identified no factual inaccuracies, guideline inconsistencies, or potentially harmful statements. Emotional prompts did, however, markedly increase relational linguistic density almost all primary prompt outputs had a CRI of zero, whereas emotional prompt outputs showed frequent use of modal verbs, affirmatives, and empathetic expressions with index values extending up to 5.28.

ChatGPT 4.0 Omni can generate Polish language discharge instructions after laparoscopic cholecystectomy that are accurate and largely complete for core postoperative domains, but with important gaps in antibiotic related information and structured dietary guidance. Emotional prompting substantially enhances the relational and motivational tone without improving content completeness. Nurse led discharge counselling and clinician review therefore remain essential when using large language model generated materials, ideally supported by standardised prompts, ERAS aligned benchmarks, and human in the loop oversight.