Comparative efficiency of Asian surgical systems in COVID-19 response: a multi-level longitudinal benchmarking study

Surgical systems in Southeast Asia were already experiencing significant supply shortages before the COVID-19 pandemic, which further exacerbated these deficits. With the aim of identifying modifiable factors for improvement, our study benchmarks surgical system efficiencies of eight Southeast Asian Hospital and Health Systems' (HHS) in combating COVID-19, with an analysis spanning three levels: country, city and HHS.

Based on 18-month data (January 2020 to June 2021), we developed a two-stage frontier benchmarking approach that applies efficiency benchmarking techniques to evaluate how close organisations operate relative to an efficiency frontier (representing the maximum potential output achievable with a given set of inputs). Stage 1 involves data envelopment analysis, which yields monthly national efficiency scores for five countries based on country-level panel data. Stage 2 involves stochastic frontier analysis, followed by evaluating HHS-level efficiency with HHS-level data over 64 possible model configurations, adjusting for stage 1 national-level efficiency scores as well as city-level responses.

Among the 64 possible stage 2 model configurations, two model clusters comprising 36 plausible models demonstrated superior fit to the observed data, thereby providing robust insights through majority voting. Policy measures related to access control and workplace closure were positively associated with recovery rates. Elective surgery volumes showed a negative association and emergency surgery volumes a positive association with efficiency. Our analysis indicates lags of 0-1 month between input changes and effects on the surgical system.

Insights from the benchmarking of HHS efficiencies will help inform surgical system policy and responses. Access control and workplace closure policies are associated with COVID-19 recovery rates. Emergency surgery volumes are associated with higher efficiency and elective surgery volumes with lower efficiency. System responses to policy measures can manifest with lags of up to a month. Considering insights from multiple plausible models makes these conclusions more robust.