Current evidence and future directions of metrics used to evaluate ambient clinical documentation: A scoping review

Although clinical documentation serves a vital role in healthcare, this task generates a significant administrative burden and has been linked to rising rates of clinician burnout. Ambient clinical documentation (ACD) provides a potential solution to this issue. With the ongoing developments in this field, benchmarking and evaluation of ACD as the technology evolves is necessary, and to facilitate this a standardised set of evaluation criteria is needed. This review aims to evaluate the available literature on the use of ACD technology, identify and analyse the outcome measures used to evaluate ACD technology, and propose a framework for evaluating ACD.

A comprehensive search of Medline and Embase databases was conducted to identify primary studies reporting on the use of ACD technology, defined as systems that process simulated or real-world clinician-patient conversations to generate structured clinical notes.

11 studies were included in the final review. Four main themes to evaluate ACD technology were identified, including model performance, documentation efficiency, clinician experience, and patient experience. However, there was significant heterogeneity of the evaluation metrics used among the included studies. Following this, the review proposes the Ambient Clinical Documentation Quality Instrument (ACDQI) as a set of evaluation metrics to assess the performance of ACD systems.

ACD technology has the potential to significantly reduce documentation burden for clinicians, as well as redirect saved time to direct patient care. Further collaborative work is needed to determine appropriate amendments to the proposed ACDQI scoring system prior to validation and implementation in future ACD studies.