Evaluation of a chat GPT generated patient information leaflet about laparoscopic cholecystectomy

Artificial intelligence is increasingly being used in all aspects of life in information compilation and writing, and this includes healthcare. This study aimed to evaluate a Chat GPT generated patient information leaflet (PIL) against a surgeon generated version, in order to explore a potential application of this artificial intelligence language processing model.

Cross-sectional study, undertaken May to June 2023, asking two cohorts (patients and doctors) to complete a questionnaire evaluating a Chat GPT generated PIL and a surgeon generated PIL about laparoscopic cholecystectomy. The patients were having laparoscopic cholecystectomy at large private Hospital in Melbourne, Australia, and doctors were recruited from this hospital and a public quaternary hospital in Melbourne, Australia. The study included a convenience sample of 28 patients and 16 doctors. The main outcome measure was a questionnaire (maximum score out of 8) based on validated evaluation instrument for PILs.

The study recruited 28 patients and 15 doctors to complete the questionnaire. The Chat GPT and surgeon generated PILs were scored similarly by patients (median 8 for both PIL; mean 7.5 for Chat GPT PIL vs. 7.1 for surgeon PIL). Doctors also scored both versions similarly, with slightly higher scores for Chat GPT over surgeon version (median 7 vs. 6; mean 6.7 vs. 5.6, respectively).

The Chat GPT generated PIL was assessed as being as good or slightly better than the surgeon generated version. This study shows that PIL are a feasible application of AI language processing models.