AI-Generated Draft Response Tool, Message Turnaround Time, and After-Hours Replies Among Health Care Professionals

Patient portal message volumes have increased, creating substantial administrative burden for clinicians and contributing to after-hours electronic health record work. The Epic In Basket Augmented Response Technology tool generates artificial intelligence-drafted replies to patient messages and has been widely deployed to reduce this burden, yet assessment of efficiency remains limited.

To examine the associations of response tool adoption, use, and habituation with message turnaround time and after-hours response.

This retrospective cohort study was conducted using clinician-level panel regression with individual fixed effects. All first responses to patient portal messages sent at Columbia University Irving Medical Center (CUIMC) between March 1, 2023, and May 15, 2025, were analyzed.

Enterprise-level response tool deployment December 2024.

The primary outcomes were log-transformed message turnaround time, defined as the elapsed time between patient message submission and first response sent by a clinician, and whether the response was sent outside working hours.

The analysis included 983 154 responses from 1542 clinicians (770 [49.9%] female) and 272 091 patients (mean [SD] age, 48.1 [21.6] years; 121 370 [44.6%] female) at CUIMC. Response tool adoption was associated with an 8.6% (95% CI, 4.6%-12.9%; P < .001) increase in overall message turnaround time. Using the tool was associated with a significant reduction in turnaround time (-29.5% [95% CI, -34.5% to -24.1%]; P < .001). Turnaround time findings were robust across sensitivity analyses. Response tool use was not associated with an overall reduction of after-hours messaging burden (0.61 [95% CI, -1.02 to 2.24] percentage points; P = .46). However, for test results-related queries received during working hours, response tool use was associated with a reduction in after-hours responses (-3.67 [95% CI, -6.94 to -0.39] percentage points; P = .03), and for all messages received after hours, response tool use was associated with an increase in after-hours responses (4.52 [95% CI, 1.54-7.50] percentage points; P = .003). These after-hours findings were sensitive to the definition of working hours in sensitivity analyses.

In this cohort study, response tool adoption was associated with increased turnaround time. Response tool use was associated with reduced turnaround time and with fewer after-hours responses for test results-related queries received during working hours, but more after-hours responses for messages received outside working hours. These findings suggest that the tool may reduce the barrier to responding to certain messages while not uniformly reducing burden.