Assessing the Efficacy and Clinical Utility of Artificial Intelligence Scribes in Urology

To assess the quality of artificial intelligence (AI) scribes and to evaluate their impact on urologic practice.

Standardized reference consultation notes were created for common urologic referrals (urolithiasis, benign prostate hyperplasia, and prostate-specific antigen screening) were created. Audio recordings of these simulated patient encounters were played for five freely accessible AI scribes. The outputs generated by these scribes were evaluated using a standardized survey completed by Canadian urology faculty and trainees across two academic sites.

Twenty urologists responded, of which 75% reported clinical documentation as a significant source of burnout. 90% expressed openness to using AI scribes to facilitate documentation. Among the AI tools assessed, Nabla was ranked the most effective, with a favorable composite score of 68% and lowest critical error composite score (28%). Three-quarters of respondents indicated a willingness to implement AI scribes in their current form, and 89% believed AI scribes would significantly alter their practice in the future.

AI scribes have the potential to substantially alleviate the administrative burden associated with clinical documentation, thereby reducing burnout. However, challenges remain, particularly in ensuring accuracy, and addressing medicolegal and privacy concerns. AI scribes represent a valuable tool for reducing the documentation burden and improving patient-clinician interactions. However, they should complement, rather than replace, clinician-led documentation to maintain the quality and safety of patient care.