Feasibility of Automated Precharting using GPT-4 in New Specialty Referrals

This study evaluates the feasibility of using GPT-4 to automate precharting for specialty referrals, focusing on new patients referred to an otolaryngology clinic for nasal congestion. We describe the design decisions and strategies tested in creating this precharting utility, including methods for prompt design and token limit handling. Through iterative testing and building, our tool achieved 95.0% agreement with physician consensus in a small retrospective test sample. Results from a small prospective pilot showed favorable feedback of summaries in a real-world clinical setting, though there was a discrepancy between high intention to use the summary but lower perception of time savings. Our results demonstrate that automated pre-charting with accuracy and clinical relevance can be feasible with large language models such as GPT-4. Our design features can inform the development of vendor chart summarization solutions.