Quality and sustainability of Ethiopia's national surgical indicators

In 2015, the Ethiopian Federal Ministry of Health (FMOH) developed the Saving Lives through Safe Surgery (SaLTS) initiative to improve national surgical care. Previous work led to development and implementation of 15 surgical key performance indicators (KPIs) to standardize surgical data practices. The objective of this project is to investigate current practices of KPI data collection and assess quality to improve data management and strengthen surgical systems. The first portion of the study documented the surgical data collection process including methods, instruments, and effectiveness at 10 hospitals across 2 regions in Ethiopia. Secondly, data for KPIs of focus [1. Surgical Volume, 2. Perioperative Mortality Rate (POMR), 3. Adverse Anesthetic Outcome (AAO), 4. Surgical Site Infection (SSI), and 5. Safe Surgery Checklist (SSC) Utilization] were compared between registries, KPI reporting forms, and the DHIS2 (district health information system) electronic database for a 6-month period (January-June 2022). Quality was assessed based on data completeness and consistency. The data collection process involved hospital staff recording data elements in registries, quality officers calculating KPIs, completing monthly KPI reporting forms, and submitting data into DHIS2 for the national and regional health bureaus. Data quality verifications revealed discrepancies in consistency at all hospitals, ranging from 1-3 indicators. For all hospitals, average monthly surgical volume was 57 cases, POMR was 0.38% (13/3399), inpatient SSI rate was 0.79% (27/3399), AAO rate was 0.15% (5/3399), and mean SSC utilization monthly was 93% (100% median). Half of the hospitals had incomplete data within the registries, ranging from 2-5 indicators. AAO, SSC, and SSI were commonly missing data in registries. Non-standardized KPI reporting forms contributed significantly to the findings. Facilitators to quality data collection included continued use of registries from previous interventions and use of a separate logbook to document specific KPIs. Delayed rollout of these indicators in each region contributed to issues in data quality. Barriers involved variable indicator recording from different personnel, data collection tools that generate false positives (i.e. completeness of SSC defined as paper form filled out prior to patient discharge) or missing data because of reporting time period (i.e. monthly SSI may miss infections outside of one month), inadequate data elements in registries, and lack of standardized monthly KPI reporting forms. As the FMOH introduces new indicators and changes, we recommend continuous and consistent quality checks and data capacity building, including the use of routinely generated health information for quality improvement projects at the department level.