Exploring the Potential of ChatGPT for the Summarization of Patient Medical Histories: A Pilot Study

Improvements in operations through the use of artificial intelligence (AI) are expected in various fields. Chat Generative Pre-trained Transformer (ChatGPT), released in November 2022, has gained rapid popularity and is widely used; however, from a privacy perspective, research on the use of AI in medicine is limited. In this study, we compared 68 discharge summaries generated by ChatGPT based on interim summaries (hereafter, "AI summaries") with those generated by junior residents (hereafter, "resident summaries") and investigated the challenges and benefits of using ChatGPT. Resident summaries significantly outperformed AI summaries in quality (p = 0.011), factuality (p < 0.002), and completeness (p < 0.001) but not in readability (p = 0.08). For hospitalizations exceeding three months (n = 12), no significant differences were observed between resident and AI summaries on any measure. These findings suggest that AI can produce discharge summaries comparable to those of residents in complex, long-term cases. Thus, responsible use of ChatGPT, ensuring data privacy, can aid in efficient discharge summary creation.